2024-08-15T16:46:56.458 INFO:root:teuthology version: 1.2.0 2024-08-15T16:46:56.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T16:46:56.567 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: mchangir@redhat.com first_in_suite: false job_id: '7856960' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5995 sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 targets: smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-15_16:34:53 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-15T16:46:56.567 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa; will attempt to use it 2024-08-15T16:46:56.568 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks 2024-08-15T16:46:56.568 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-15T16:46:56.570 INFO:teuthology.task.internal:Checking packages... 2024-08-15T16:46:56.592 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2c991c196a651f8ef0f36c7763ec87a241b25549' 2024-08-15T16:46:56.592 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-15T16:46:56.592 INFO:teuthology.packaging:ref: None 2024-08-15T16:46:56.592 INFO:teuthology.packaging:tag: None 2024-08-15T16:46:56.592 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T16:46:56.592 INFO:teuthology.packaging:sha1: 2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T16:46:56.592 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-15T16:46:56.741 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375-g2c991c19-1focal 2024-08-15T16:46:56.742 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-15T16:46:56.750 INFO:teuthology.task.internal:no buildpackages task found 2024-08-15T16:46:56.750 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-15T16:46:56.794 INFO:teuthology.task.internal:Saving configuration 2024-08-15T16:46:56.812 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-15T16:46:56.827 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-15T16:46:56.851 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 16:40:33.784477', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T16:46:56.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-15 16:40:33.782831', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-08-15T16:46:56.873 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-15T16:46:56.880 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-08-15T16:46:56.880 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-08-15T16:46:56.880 INFO:teuthology.run_tasks:Running task console_log... 2024-08-15T16:46:56.956 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f76ed01bf40>, signals=[15]) 2024-08-15T16:46:56.956 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-15T16:46:56.963 INFO:teuthology.task.internal:Opening connections... 2024-08-15T16:46:56.963 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:46:56.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:46:57.062 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:46:57.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:46:57.146 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-15T16:46:57.153 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-08-15T16:46:57.160 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-08-15T16:46:57.160 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-08-15T16:46:57.210 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="Ubuntu" 2024-08-15T16:46:57.210 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T16:46:57.210 INFO:teuthology.orchestra.run.smithi146.stdout:ID=ubuntu 2024-08-15T16:46:57.210 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE=debian 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="20.04" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_CODENAME=focal 2024-08-15T16:46:57.211 INFO:teuthology.orchestra.run.smithi146.stdout:UBUNTU_CODENAME=focal 2024-08-15T16:46:57.212 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-08-15T16:46:57.232 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-08-15T16:46:57.239 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-08-15T16:46:57.239 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="20.04" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-08-15T16:46:57.285 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=focal 2024-08-15T16:46:57.286 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=focal 2024-08-15T16:46:57.286 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-08-15T16:46:57.307 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-15T16:46:57.316 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-15T16:46:57.334 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-15T16:46:57.335 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-08-15T16:46:57.337 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-08-15T16:46:57.343 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-15T16:46:57.360 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-15T16:46:57.360 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-08-15T16:46:57.362 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-08-15T16:46:57.396 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-15T16:46:57.450 INFO:teuthology.run_tasks:Running task kernel... 2024-08-15T16:46:57.464 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-15T16:46:57.464 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-15T16:46:57.464 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T16:46:57.465 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-15T16:46:57.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:46:57.470 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-08-15T16:46:57.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:46:57.471 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-08-15T16:46:57.477 INFO:teuthology.orchestra.run.smithi146.stdout:5.4.0-124-generic 2024-08-15T16:46:57.477 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.4.0-124-generic 2024-08-15T16:46:57.477 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get clean 2024-08-15T16:46:57.521 INFO:teuthology.orchestra.run.smithi161.stdout:5.4.0-124-generic 2024-08-15T16:46:57.521 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.4.0-124-generic 2024-08-15T16:46:57.521 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2024-08-15T16:46:57.580 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-08-15T16:46:57.633 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-08-15T16:46:57.781 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:46:57.794 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T16:46:57.859 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T16:46:57.869 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:46:57.885 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T16:46:57.895 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-08-15T16:46:57.934 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-08-15T16:46:58.155 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T16:46:58.235 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T16:46:58.273 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T16:46:58.278 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T16:46:58.278 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T16:46:58.287 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-08-15T16:46:58.302 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T16:46:58.302 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T16:46:58.306 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T16:46:58.306 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T16:46:58.314 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T16:46:58.319 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T16:46:58.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T16:46:58.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T16:46:58.322 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T16:46:58.326 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T16:46:58.327 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T16:46:58.327 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T16:46:58.328 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T16:46:58.344 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T16:46:58.344 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T16:46:58.344 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T16:46:58.345 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T16:46:58.388 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T16:46:58.456 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T16:46:58.525 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,016 kB] 2024-08-15T16:46:58.745 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,125 kB] 2024-08-15T16:46:58.867 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,495 kB] 2024-08-15T16:46:58.936 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T16:46:58.951 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T16:46:58.951 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T16:46:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [796 kB] 2024-08-15T16:46:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [464 kB] 2024-08-15T16:46:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.1 kB] 2024-08-15T16:46:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,034 kB] 2024-08-15T16:46:59.041 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T16:46:59.041 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T16:46:59.050 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [543 kB] 2024-08-15T16:46:59.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T16:46:59.054 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T16:46:59.063 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-08-15T16:46:59.063 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,152 kB] 2024-08-15T16:46:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-08-15T16:46:59.066 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [425 kB] 2024-08-15T16:46:59.073 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [544 B] 2024-08-15T16:46:59.073 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T16:46:59.081 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [674 kB] 2024-08-15T16:46:59.085 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [999 kB] 2024-08-15T16:46:59.099 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T16:46:59.106 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T16:46:59.107 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T16:46:59.107 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T16:46:59.136 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [212 kB] 2024-08-15T16:46:59.137 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.1 kB] 2024-08-15T16:46:59.139 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-08-15T16:46:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-08-15T16:46:59.153 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-08-15T16:46:59.154 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [442 kB] 2024-08-15T16:46:59.165 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [540 B] 2024-08-15T16:46:59.165 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,219 kB] 2024-08-15T16:46:59.185 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T16:46:59.186 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T16:46:59.208 INFO:teuthology.orchestra.run.smithi146.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [801 kB] 2024-08-15T16:46:59.215 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-08-15T16:46:59.215 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-08-15T16:46:59.227 INFO:teuthology.orchestra.run.smithi146.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [294 kB] 2024-08-15T16:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [27.8 kB] 2024-08-15T16:46:59.235 INFO:teuthology.orchestra.run.smithi146.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-08-15T16:46:59.236 INFO:teuthology.orchestra.run.smithi146.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-08-15T16:46:59.295 INFO:teuthology.orchestra.run.smithi146.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-08-15T16:46:59.296 INFO:teuthology.orchestra.run.smithi146.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-08-15T16:46:59.296 INFO:teuthology.orchestra.run.smithi146.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-08-15T16:46:59.298 INFO:teuthology.orchestra.run.smithi146.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-08-15T16:46:59.299 INFO:teuthology.orchestra.run.smithi146.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-08-15T16:46:59.377 INFO:teuthology.orchestra.run.smithi146.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-08-15T16:46:59.378 INFO:teuthology.orchestra.run.smithi146.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-08-15T16:46:59.378 INFO:teuthology.orchestra.run.smithi146.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-08-15T16:47:02.697 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 21.4 MB in 3s (7,262 kB/s) 2024-08-15T16:47:03.124 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 21.4 MB in 3s (6,166 kB/s) 2024-08-15T16:47:04.153 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:47:04.179 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T16:47:04.248 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:47:04.441 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-08-15T16:47:04.442 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-08-15T16:47:04.580 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:47:04.605 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T16:47:04.632 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-08-15T16:47:04.633 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T16:47:04.633 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T16:47:04.633 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T16:47:04.634 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-08-15T16:47:04.634 INFO:teuthology.orchestra.run.smithi161.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T16:47:04.634 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2024-08-15T16:47:04.634 INFO:teuthology.orchestra.run.smithi161.stdout: thermald 2024-08-15T16:47:04.680 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:47:04.688 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-08-15T16:47:04.689 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T16:47:04.689 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T16:47:04.689 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T16:47:04.690 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-08-15T16:47:04.691 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T16:47:04.874 INFO:teuthology.orchestra.run.smithi161.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T16:47:04.874 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 77.2 MB of archives. 2024-08-15T16:47:04.874 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T16:47:04.874 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T16:47:04.894 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-08-15T16:47:04.895 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-08-15T16:47:05.091 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-08-15T16:47:05.091 INFO:teuthology.orchestra.run.smithi146.stdout: linux-generic linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T16:47:05.091 INFO:teuthology.orchestra.run.smithi146.stdout: linux-headers-generic linux-image-5.4.0-192-generic 2024-08-15T16:47:05.092 INFO:teuthology.orchestra.run.smithi146.stdout: linux-modules-5.4.0-192-generic linux-modules-extra-5.4.0-192-generic 2024-08-15T16:47:05.093 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-08-15T16:47:05.093 INFO:teuthology.orchestra.run.smithi146.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-08-15T16:47:05.093 INFO:teuthology.orchestra.run.smithi146.stdout:Recommended packages: 2024-08-15T16:47:05.093 INFO:teuthology.orchestra.run.smithi146.stdout: thermald 2024-08-15T16:47:05.158 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-08-15T16:47:05.158 INFO:teuthology.orchestra.run.smithi146.stdout: linux-headers-5.4.0-192 linux-headers-5.4.0-192-generic 2024-08-15T16:47:05.158 INFO:teuthology.orchestra.run.smithi146.stdout: linux-image-5.4.0-192-generic linux-modules-5.4.0-192-generic 2024-08-15T16:47:05.158 INFO:teuthology.orchestra.run.smithi146.stdout: linux-modules-extra-5.4.0-192-generic 2024-08-15T16:47:05.160 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-08-15T16:47:05.161 INFO:teuthology.orchestra.run.smithi146.stdout: linux-generic linux-headers-generic linux-image-generic 2024-08-15T16:47:05.214 INFO:teuthology.orchestra.run.smithi146.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T16:47:05.214 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 77.2 MB of archives. 2024-08-15T16:47:05.214 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 381 MB of additional disk space will be used. 2024-08-15T16:47:05.214 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-192-generic amd64 5.4.0-192.212 [15.0 MB] 2024-08-15T16:47:05.613 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T16:47:05.711 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T16:47:06.108 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-192-generic amd64 5.4.0-192.212 [10.5 MB] 2024-08-15T16:47:06.176 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T16:47:06.176 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T16:47:06.176 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T16:47:06.267 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T16:47:06.278 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T16:47:06.425 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-192-generic amd64 5.4.0-192.212 [39.2 MB] 2024-08-15T16:47:06.925 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 77.2 MB in 1s (69.5 MB/s) 2024-08-15T16:47:07.095 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T16:47:07.719 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.192.190 [1,900 B] 2024-08-15T16:47:07.720 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.192.190 [2,588 B] 2024-08-15T16:47:07.720 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192 all 5.4.0-192.212 [11.0 MB] 2024-08-15T16:47:07.999 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-192-generic amd64 5.4.0-192.212 [1,398 kB] 2024-08-15T16:47:08.013 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.192.190 [2,456 B] 2024-08-15T16:47:08.744 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 77.2 MB in 3s (23.3 MB/s) 2024-08-15T16:47:08.807 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T16:47:08.812 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:08.900 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:08.969 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-5.4.0-192-generic. 2024-08-15T16:47:10.635 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-08-15T16:47:10.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../0-linux-modules-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:10.733 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:10.909 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T16:47:10.922 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:11.040 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:12.078 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T16:47:12.093 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:12.127 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:12.616 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-image-5.4.0-192-generic. 2024-08-15T16:47:12.629 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../1-linux-image-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:12.729 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:13.843 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-192-generic. 2024-08-15T16:47:13.856 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:13.884 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:16.679 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:16.839 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:17.184 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:17.309 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:17.645 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T16:47:17.659 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T16:47:17.703 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T16:47:18.561 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../3-linux-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:18.687 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:19.013 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:19.140 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:19.493 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-5.4.0-192. 2024-08-15T16:47:19.508 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../5-linux-headers-5.4.0-192_5.4.0-192.212_all.deb ... 2024-08-15T16:47:19.551 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T16:47:23.187 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T16:47:23.208 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:23.245 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:25.586 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-5.4.0-192-generic. 2024-08-15T16:47:25.608 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../6-linux-headers-5.4.0-192-generic_5.4.0-192.212_amd64.deb ... 2024-08-15T16:47:25.652 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:25.969 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:26.091 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:26.431 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T16:47:26.578 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:27.780 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:28.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.192.190_amd64.deb ... 2024-08-15T16:47:28.776 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-generic (5.4.0.192.190) over (5.4.0.124.125) ... 2024-08-15T16:47:28.932 INFO:teuthology.orchestra.run.smithi146.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:47:28.932 INFO:teuthology.orchestra.run.smithi146.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:29.057 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:29.116 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-5.4.0-192 (5.4.0-192.212) ... 2024-08-15T16:47:29.183 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:29.239 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:30.401 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T16:47:30.441 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:30.527 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T16:47:30.653 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T16:47:30.780 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:30.868 INFO:teuthology.orchestra.run.smithi146.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T16:47:30.868 INFO:teuthology.orchestra.run.smithi146.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:47:31.586 INFO:teuthology.orchestra.run.smithi161.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:31.735 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:31.861 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-extra-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:33.037 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-generic (5.4.0.192.190) ... 2024-08-15T16:47:33.196 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-generic (5.4.0.192.190) ... 2024-08-15T16:47:33.322 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-generic (5.4.0.192.190) ... 2024-08-15T16:47:33.449 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for linux-image-5.4.0-192-generic (5.4.0-192.212) ... 2024-08-15T16:47:33.514 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-08-15T16:47:33.515 INFO:teuthology.orchestra.run.smithi161.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:53.320 INFO:teuthology.orchestra.run.smithi146.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T16:47:53.320 INFO:teuthology.orchestra.run.smithi146.stdout:Sourcing file `/etc/default/grub' 2024-08-15T16:47:53.328 INFO:teuthology.orchestra.run.smithi146.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T16:47:53.339 INFO:teuthology.orchestra.run.smithi146.stdout:Generating grub configuration file ... 2024-08-15T16:47:53.657 INFO:teuthology.orchestra.run.smithi146.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:47:53.696 INFO:teuthology.orchestra.run.smithi146.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:54.108 INFO:teuthology.orchestra.run.smithi146.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T16:47:54.120 INFO:teuthology.orchestra.run.smithi146.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T16:47:55.789 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-08-15T16:47:55.790 INFO:teuthology.orchestra.run.smithi161.stdout:Sourcing file `/etc/default/grub' 2024-08-15T16:47:55.800 INFO:teuthology.orchestra.run.smithi161.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T16:47:55.815 INFO:teuthology.orchestra.run.smithi161.stdout:Generating grub configuration file ... 2024-08-15T16:47:56.177 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:47:56.198 INFO:teuthology.orchestra.run.smithi146.stdout:done 2024-08-15T16:47:56.202 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:47:56.580 DEBUG:teuthology.orchestra.run.smithi146:> dpkg -s linux-image-generic 2024-08-15T16:47:56.585 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T16:47:56.596 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T16:47:56.634 INFO:teuthology.orchestra.run.smithi146.stdout:Package: linux-image-generic 2024-08-15T16:47:56.634 INFO:teuthology.orchestra.run.smithi146.stdout:Status: install ok installed 2024-08-15T16:47:56.634 INFO:teuthology.orchestra.run.smithi146.stdout:Priority: optional 2024-08-15T16:47:56.634 INFO:teuthology.orchestra.run.smithi146.stdout:Section: kernel 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Installed-Size: 20 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: amd64 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Source: linux-meta 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Version: 5.4.0.192.190 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Recommends: thermald 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout:Description: Generic Linux kernel image 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout: This package will always depend on the latest generic kernel image 2024-08-15T16:47:56.635 INFO:teuthology.orchestra.run.smithi146.stdout: available. 2024-08-15T16:47:56.636 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T16:47:56.636 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T16:47:56.636 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T16:47:56.636 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-08-15T16:47:56.636 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T16:47:56.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get clean 2024-08-15T16:47:56.735 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-08-15T16:47:56.863 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T16:47:56.982 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:47:57.064 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T16:47:57.145 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T16:47:58.413 INFO:teuthology.orchestra.run.smithi161.stdout:done 2024-08-15T16:47:58.718 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:47:58.745 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T16:47:58.817 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:47:58.864 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2024-08-15T16:47:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2024-08-15T16:47:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 20 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 5.4.0.192.190 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2024-08-15T16:47:58.922 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2024-08-15T16:47:58.923 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-192-generic 2024-08-15T16:47:58.923 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-15T16:47:58.923 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-15T16:47:58.923 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-08-15T16:47:58.923 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2024-08-15T16:47:58.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2024-08-15T16:47:59.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-08-15T16:47:59.029 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-08-15T16:47:59.030 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-08-15T16:47:59.147 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:47:59.153 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T16:47:59.171 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T16:47:59.260 INFO:teuthology.orchestra.run.smithi146.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T16:47:59.261 INFO:teuthology.orchestra.run.smithi146.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T16:47:59.262 DEBUG:teuthology.orchestra.run.smithi146:> dpkg -s linux-image-generic 2024-08-15T16:47:59.266 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Package: linux-image-generic 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Status: install ok installed 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Priority: optional 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Section: kernel 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Installed-Size: 20 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: amd64 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Source: linux-meta 2024-08-15T16:47:59.282 INFO:teuthology.orchestra.run.smithi146.stdout:Version: 5.4.0.192.190 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout:Recommends: thermald 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout:Description: Generic Linux kernel image 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout: This package will always depend on the latest generic kernel image 2024-08-15T16:47:59.283 INFO:teuthology.orchestra.run.smithi146.stdout: available. 2024-08-15T16:47:59.283 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-08-15T16:47:59.290 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.Hz2Yc0iOwg 2024-08-15T16:47:59.290 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Hz2Yc0iOwg 2024-08-15T16:47:59.299 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.Hz2Yc0iOwg 2024-08-15T16:47:59.404 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.Hz2Yc0iOwg is 9KB 2024-08-15T16:47:59.418 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.Hz2Yc0iOwg 2024-08-15T16:47:59.425 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T16:47:59.435 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:47:59.435 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T16:47:59.436 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T16:47:59.510 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T16:47:59.510 DEBUG:teuthology.orchestra.run.smithi146:> sudo update-grub 2024-08-15T16:48:00.830 INFO:teuthology.orchestra.run.smithi146.stderr:Sourcing file `/etc/default/grub' 2024-08-15T16:48:00.839 INFO:teuthology.orchestra.run.smithi146.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T16:48:00.854 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-08-15T16:48:01.020 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:48:01.046 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-08-15T16:48:01.099 INFO:teuthology.orchestra.run.smithi146.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:48:01.111 INFO:teuthology.orchestra.run.smithi146.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:48:01.115 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:48:01.307 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-08-15T16:48:01.307 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-08-15T16:48:01.472 INFO:teuthology.orchestra.run.smithi146.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T16:48:01.484 INFO:teuthology.orchestra.run.smithi146.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T16:48:01.536 INFO:teuthology.orchestra.run.smithi161.stdout:linux-image-generic is already the newest version (5.4.0.192.190). 2024-08-15T16:48:01.536 INFO:teuthology.orchestra.run.smithi161.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-08-15T16:48:01.538 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 20 2024-08-15T16:48:01.559 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 5.4.0.192.190 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-5.4.0-192-generic, linux-modules-extra-5.4.0-192-generic, linux-firmware, intel-microcode, amd64-microcode 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2024-08-15T16:48:01.560 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2024-08-15T16:48:01.560 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-08-15T16:48:01.565 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.Cfy8zgwiAO 2024-08-15T16:48:01.565 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Cfy8zgwiAO 2024-08-15T16:48:01.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.Cfy8zgwiAO 2024-08-15T16:48:01.667 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.Cfy8zgwiAO is 9KB 2024-08-15T16:48:01.679 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.Cfy8zgwiAO 2024-08-15T16:48:01.685 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-08-15T16:48:01.745 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:48:01.746 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-08-15T16:48:01.746 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-08-15T16:48:01.780 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-192-generic 2024-08-15T16:48:01.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo update-grub 2024-08-15T16:48:02.548 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-08-15T16:48:02.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-08-15T16:48:03.148 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub' 2024-08-15T16:48:03.152 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-08-15T16:48:03.168 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-08-15T16:48:03.431 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.4.0-192-generic 2024-08-15T16:48:03.443 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.4.0-192-generic 2024-08-15T16:48:03.845 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-08-15T16:48:03.858 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-08-15T16:48:04.649 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-08-15T16:48:04.651 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-08-15T16:48:32.580 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-08-15T16:48:32.581 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:48:32.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:48:34.655 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-08-15T16:48:34.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:48:34.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:48:50.911 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-08-15T16:48:53.218 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-08-15T16:48:59.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:48:59.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:49:02.220 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:49:02.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:49:02.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-08-15T16:49:05.279 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-08-15T16:49:14.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:49:14.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:49:17.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:49:17.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:49:33.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-08-15T16:49:48.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:49:48.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:49:48.830 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-08-15T16:49:49.721 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:49:49.721 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-192-generic"... 2024-08-15T16:49:49.721 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-08-15T16:49:49.727 INFO:teuthology.orchestra.run.smithi146.stdout:5.4.0-192-generic 2024-08-15T16:49:49.727 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T16:49:49.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T16:49:49.727 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T16:49:50.728 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-15T16:49:50.728 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T16:49:50.819 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-08-15T16:49:50.840 DEBUG:teuthology.parallel:result is None 2024-08-15T16:50:17.284 DEBUG:teuthology.orchestra.remote:timed out 2024-08-15T16:50:32.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:50:32.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:50:32.541 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-08-15T16:50:33.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:50:33.173 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-192-generic"... 2024-08-15T16:50:33.174 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-08-15T16:50:33.180 INFO:teuthology.orchestra.run.smithi161.stdout:5.4.0-192-generic 2024-08-15T16:50:33.180 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-192-generic vs 5.4.0-192-generic 2024-08-15T16:50:33.180 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-15T16:50:33.180 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-08-15T16:50:34.181 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-15T16:50:34.181 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-15T16:50:34.286 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2024-08-15T16:50:34.308 DEBUG:teuthology.parallel:result is None 2024-08-15T16:50:34.308 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-15T16:50:34.316 INFO:teuthology.task.internal:Creating test directory... 2024-08-15T16:50:34.316 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T16:50:34.319 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-15T16:50:34.326 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-15T16:50:34.351 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-15T16:50:34.486 INFO:teuthology.task.internal:Creating archive directory... 2024-08-15T16:50:34.487 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T16:50:34.489 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-15T16:50:34.513 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-15T16:50:34.526 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-15T16:50:34.526 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T16:50:34.544 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-15T16:50:34.562 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T16:50:34.568 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T16:50:34.573 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T16:50:34.580 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-15T16:50:34.582 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-15T16:50:34.589 INFO:teuthology.task.internal:Configuring sudo... 2024-08-15T16:50:34.589 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T16:50:34.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-15T16:50:34.607 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-15T16:50:34.618 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-15T16:50:34.618 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T16:50:34.652 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-15T16:50:34.658 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T16:50:34.664 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T16:50:34.672 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:50:34.672 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T16:50:34.693 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-15T16:50:34.700 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-15T16:50:34.749 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:50:34.749 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-15T16:50:34.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-08-15T16:50:34.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-08-15T16:50:34.829 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-15T16:50:34.837 INFO:teuthology.task.internal:Starting timer... 2024-08-15T16:50:34.837 INFO:teuthology.run_tasks:Running task pcp... 2024-08-15T16:50:34.855 INFO:teuthology.run_tasks:Running task selinux... 2024-08-15T16:50:34.886 DEBUG:teuthology.task.selinux:Excluding smithi146: OS 'ubuntu' does not support SELinux 2024-08-15T16:50:34.887 DEBUG:teuthology.task.selinux:Excluding smithi161: OS 'ubuntu' does not support SELinux 2024-08-15T16:50:34.887 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-15T16:50:34.887 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-08-15T16:50:34.887 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-15T16:50:34.887 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-15T16:50:34.922 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-15T16:50:34.925 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-15T16:50:35.030 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-15T16:50:35.041 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-15T16:50:35.042 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi146.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-15T16:56:04.525 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-08-15T16:56:04.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:56:04.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:56:04.616 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-08-15T16:56:04.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-08-15T16:56:04.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:56:04.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-15T16:56:04.846 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-08-15T16:56:04.968 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-08-15T16:56:04.968 INFO:teuthology.run_tasks:Running task clock... 2024-08-15T16:56:04.980 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-15T16:56:04.980 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T16:56:04.981 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T16:56:04.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-15T16:56:04.984 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T16:56:05.018 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T16:56:05.018 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Command line: ntpd -gq 2024-08-15T16:56:05.019 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-08-15T16:56:05.019 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Command line: ntpd -gq 2024-08-15T16:56:05.019 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: proto: precision = 0.093 usec (-23) 2024-08-15T16:56:05.020 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: proto: precision = 0.300 usec (-22) 2024-08-15T16:56:05.020 INFO:teuthology.orchestra.run.smithi146.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T16:56:05.021 INFO:teuthology.orchestra.run.smithi161.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T16:56:05.021 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T16:56:05.021 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-08-15T16:56:05.022 INFO:teuthology.orchestra.run.smithi146.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T16:56:05.022 INFO:teuthology.orchestra.run.smithi161.stderr:restrict ::: KOD does nothing without LIMITED. 2024-08-15T16:56:05.023 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen normally on 3 enp3s0f1 172.21.15.146:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen normally on 4 lo [::1]:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:76fd%5]:123 2024-08-15T16:56:05.024 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:05 ntpd[13639]: Listening on routing socket on fd #22 for interface updates 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: restrict ::: KOD does nothing without LIMITED. 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen and drop on 0 v6wildcard [::]:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen normally on 2 lo 127.0.0.1:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen normally on 3 enp3s0f1 172.21.15.161:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen normally on 4 lo [::1]:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7251%5]:123 2024-08-15T16:56:05.025 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:05 ntpd[13652]: Listening on routing socket on fd #22 for interface updates 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:06.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:06.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:06 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:06.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:06 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:07.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:07.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:07.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:07.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:07.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:07.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:07 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:07.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:07.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:07.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:07.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:07.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:07.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:07 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:08.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:08 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:08.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:08.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:08 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:09 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:09.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:09.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:09 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:10.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:10.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:10 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:10.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:10.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:10 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:11.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:11.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:11.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:11.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:11.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:11.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:11 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:11.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:11.020 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:11.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:11.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:11.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:11.021 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:11 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:12.019 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:12.020 INFO:teuthology.orchestra.run.smithi146.stdout:15 Aug 16:56:12 ntpd[13639]: ntpd: time slew +0.003804 s 2024-08-15T16:56:12.021 INFO:teuthology.orchestra.run.smithi146.stdout:ntpd: time slew +0.003804s 2024-08-15T16:56:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T16:56:12.021 INFO:teuthology.orchestra.run.smithi146.stderr:15 Aug 16:56:12 ntpd[13639]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T16:56:12.022 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/rawstats.20240815: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-08-15T16:56:12.023 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/peerstats.20240815: Permission denied 2024-08-15T16:56:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:15 Aug 16:56:12 ntpd[13652]: ntpd: time slew +0.003722 s 2024-08-15T16:56:12.024 INFO:teuthology.orchestra.run.smithi161.stdout:ntpd: time slew +0.003722s 2024-08-15T16:56:12.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-08-15T16:56:12.024 INFO:teuthology.orchestra.run.smithi161.stderr:15 Aug 16:56:12 ntpd[13652]: can't open /var/log/ntpstats/loopstats.20240815: Permission denied 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.101 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================== 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.103 INFO:teuthology.orchestra.run.smithi146.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T16:56:12.104 INFO:teuthology.run_tasks:Running task install... 2024-08-15T16:56:12.113 DEBUG:teuthology.task.install:project ceph 2024-08-15T16:56:12.114 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}} 2024-08-15T16:56:12.114 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'} 2024-08-15T16:56:12.114 INFO:teuthology.task.install:Using flavor: default 2024-08-15T16:56:12.119 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-15T16:56:12.120 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-15T16:56:12.120 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-key list | grep Ceph 2024-08-15T16:56:12.121 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-key list | grep Ceph 2024-08-15T16:56:12.224 INFO:teuthology.orchestra.run.smithi146.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T16:56:12.247 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-08-15T16:56:12.284 INFO:teuthology.orchestra.run.smithi146.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T16:56:12.284 INFO:teuthology.orchestra.run.smithi146.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T16:56:12.285 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T16:56:12.285 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T16:56:12.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T16:56:12.307 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-08-15T16:56:12.307 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph.com (release key) 2024-08-15T16:56:12.308 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-08-15T16:56:12.308 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2024-08-15T16:56:12.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T16:56:12.430 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T16:56:12.430 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T16:56:12.447 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default/ 2024-08-15T16:56:12.448 INFO:teuthology.task.install.deb:Package version is 17.2.7-1375-g2c991c19-1focal 2024-08-15T16:56:12.539 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:56:12.539 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T16:56:12.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo apt-get update 2024-08-15T16:56:12.558 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:56:12.558 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-08-15T16:56:12.574 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2024-08-15T16:56:12.753 INFO:teuthology.orchestra.run.smithi146.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T16:56:12.760 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T16:56:12.775 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T16:56:12.799 INFO:teuthology.orchestra.run.smithi146.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T16:56:12.815 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal InRelease 2024-08-15T16:56:12.821 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T16:56:12.836 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-08-15T16:56:12.845 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:56:12.845 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:6 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-08-15T16:56:12.859 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal Release.gpg 2024-08-15T16:56:12.866 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T16:56:12.878 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-08-15T16:56:12.881 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-08-15T16:56:12.912 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main i386 Packages [3,369 B] 2024-08-15T16:56:12.914 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T16:56:12.960 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-08-15T16:56:12.996 INFO:teuthology.orchestra.run.smithi146.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T16:56:13.042 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-08-15T16:56:13.411 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 28.4 kB in 1s (45.7 kB/s) 2024-08-15T16:56:13.505 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 28.4 kB in 1s (36.3 kB/s) 2024-08-15T16:56:14.553 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:56:14.581 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T16:56:14.650 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:56:14.660 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:56:14.676 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1375-g2c991c19-1focal cephadm=17.2.7-1375-g2c991c19-1focal ceph-mds=17.2.7-1375-g2c991c19-1focal ceph-mgr=17.2.7-1375-g2c991c19-1focal ceph-common=17.2.7-1375-g2c991c19-1focal ceph-fuse=17.2.7-1375-g2c991c19-1focal ceph-test=17.2.7-1375-g2c991c19-1focal ceph-volume=17.2.7-1375-g2c991c19-1focal radosgw=17.2.7-1375-g2c991c19-1focal python3-rados=17.2.7-1375-g2c991c19-1focal python3-rgw=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal python3-rbd=17.2.7-1375-g2c991c19-1focal libcephfs2=17.2.7-1375-g2c991c19-1focal libcephfs-dev=17.2.7-1375-g2c991c19-1focal librados2=17.2.7-1375-g2c991c19-1focal librbd1=17.2.7-1375-g2c991c19-1focal rbd-fuse=17.2.7-1375-g2c991c19-1focal python3-cephfs=17.2.7-1375-g2c991c19-1focal cephfs-shell=17.2.7-1375-g2c991c19-1focal cephfs-top=17.2.7-1375-g2c991c19-1focal cephfs-mirror=17.2.7-1375-g2c991c19-1focal 2024-08-15T16:56:14.746 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:56:14.874 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-08-15T16:56:14.874 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-08-15T16:56:14.961 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-08-15T16:56:14.962 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-08-15T16:56:15.071 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T16:56:15.072 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T16:56:15.073 INFO:teuthology.orchestra.run.smithi161.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T16:56:15.074 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T16:56:15.075 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2024-08-15T16:56:15.076 INFO:teuthology.orchestra.run.smithi161.stdout: btrfs-tools 2024-08-15T16:56:15.155 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-08-15T16:56:15.157 INFO:teuthology.orchestra.run.smithi146.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 python3-cmd2 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T16:56:15.158 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: qttranslations5-l10n socat xmlstarlet 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-08-15T16:56:15.159 INFO:teuthology.orchestra.run.smithi146.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout:Recommended packages: 2024-08-15T16:56:15.160 INFO:teuthology.orchestra.run.smithi146.stdout: btrfs-tools 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T16:56:15.199 INFO:teuthology.orchestra.run.smithi161.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T16:56:15.200 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T16:56:15.201 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-08-15T16:56:15.202 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 librbd1 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-08-15T16:56:15.285 INFO:teuthology.orchestra.run.smithi146.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-08-15T16:56:15.286 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-08-15T16:56:15.286 INFO:teuthology.orchestra.run.smithi146.stdout: nvme-cli python-natsort-doc python-pastedeploy-tpl python3-atomicwrites 2024-08-15T16:56:15.286 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-08-15T16:56:15.286 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-08-15T16:56:15.287 INFO:teuthology.orchestra.run.smithi146.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-08-15T16:56:15.288 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-08-15T16:56:15.288 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 librbd1 2024-08-15T16:56:15.350 INFO:teuthology.orchestra.run.smithi161.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 201 MB of archives. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi146.stdout:2 upgraded, 114 newly installed, 0 to remove and 294 not upgraded. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 201 MB of archives. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 938 MB of additional disk space will be used. 2024-08-15T16:56:15.351 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T16:56:15.369 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T16:56:15.377 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T16:56:15.395 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-08-15T16:56:15.402 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T16:56:15.431 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T16:56:15.433 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T16:56:15.435 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1375-g2c991c19-1focal [3,507 kB] 2024-08-15T16:56:15.443 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T16:56:15.445 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T16:56:15.462 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T16:56:15.463 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T16:56:15.468 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T16:56:15.479 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-08-15T16:56:15.495 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T16:56:15.497 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T16:56:15.503 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T16:56:15.505 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T16:56:15.506 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T16:56:15.507 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T16:56:15.508 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T16:56:15.508 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T16:56:15.509 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T16:56:15.513 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-08-15T16:56:15.517 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T16:56:15.517 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T16:56:15.532 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T16:56:15.534 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T16:56:15.534 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T16:56:15.535 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T16:56:15.536 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T16:56:15.536 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T16:56:15.537 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T16:56:15.539 INFO:teuthology.orchestra.run.smithi146.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T16:56:15.539 INFO:teuthology.orchestra.run.smithi146.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T16:56:15.551 INFO:teuthology.orchestra.run.smithi146.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T16:56:15.553 INFO:teuthology.orchestra.run.smithi146.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T16:56:15.568 INFO:teuthology.orchestra.run.smithi146.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T16:56:15.575 INFO:teuthology.orchestra.run.smithi146.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T16:56:15.576 INFO:teuthology.orchestra.run.smithi146.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T16:56:15.577 INFO:teuthology.orchestra.run.smithi146.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T16:56:15.577 INFO:teuthology.orchestra.run.smithi146.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T16:56:15.578 INFO:teuthology.orchestra.run.smithi146.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T16:56:15.580 INFO:teuthology.orchestra.run.smithi146.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T16:56:15.581 INFO:teuthology.orchestra.run.smithi146.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T16:56:15.585 INFO:teuthology.orchestra.run.smithi146.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T16:56:15.586 INFO:teuthology.orchestra.run.smithi146.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T16:56:15.588 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T16:56:15.600 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T16:56:15.602 INFO:teuthology.orchestra.run.smithi146.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T16:56:15.603 INFO:teuthology.orchestra.run.smithi146.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T16:56:15.603 INFO:teuthology.orchestra.run.smithi146.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T16:56:15.604 INFO:teuthology.orchestra.run.smithi146.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T16:56:15.607 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T16:56:15.607 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T16:56:15.611 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T16:56:15.613 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T16:56:15.620 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T16:56:15.638 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-08-15T16:56:15.688 INFO:teuthology.orchestra.run.smithi146.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T16:56:15.703 INFO:teuthology.orchestra.run.smithi146.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T16:56:15.705 INFO:teuthology.orchestra.run.smithi146.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T16:56:15.716 INFO:teuthology.orchestra.run.smithi146.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T16:56:15.721 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T16:56:15.723 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T16:56:15.731 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T16:56:15.768 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-08-15T16:56:15.777 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-08-15T16:56:15.779 INFO:teuthology.orchestra.run.smithi146.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T16:56:15.780 INFO:teuthology.orchestra.run.smithi146.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T16:56:15.781 INFO:teuthology.orchestra.run.smithi146.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T16:56:15.781 INFO:teuthology.orchestra.run.smithi146.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T16:56:15.782 INFO:teuthology.orchestra.run.smithi146.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T16:56:15.783 INFO:teuthology.orchestra.run.smithi146.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T16:56:15.785 INFO:teuthology.orchestra.run.smithi146.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T16:56:15.786 INFO:teuthology.orchestra.run.smithi146.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T16:56:15.796 INFO:teuthology.orchestra.run.smithi146.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T16:56:15.804 INFO:teuthology.orchestra.run.smithi146.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T16:56:15.806 INFO:teuthology.orchestra.run.smithi146.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T16:56:15.809 INFO:teuthology.orchestra.run.smithi146.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T16:56:15.810 INFO:teuthology.orchestra.run.smithi146.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T16:56:15.814 INFO:teuthology.orchestra.run.smithi146.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T16:56:15.818 INFO:teuthology.orchestra.run.smithi146.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T16:56:15.818 INFO:teuthology.orchestra.run.smithi146.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T16:56:15.820 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-08-15T16:56:15.832 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-08-15T16:56:15.833 INFO:teuthology.orchestra.run.smithi146.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T16:56:15.835 INFO:teuthology.orchestra.run.smithi146.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T16:56:15.874 INFO:teuthology.orchestra.run.smithi146.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T16:56:15.875 INFO:teuthology.orchestra.run.smithi146.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T16:56:15.875 INFO:teuthology.orchestra.run.smithi146.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T16:56:15.875 INFO:teuthology.orchestra.run.smithi146.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T16:56:15.876 INFO:teuthology.orchestra.run.smithi146.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T16:56:15.876 INFO:teuthology.orchestra.run.smithi146.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T16:56:15.888 INFO:teuthology.orchestra.run.smithi146.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T16:56:15.888 INFO:teuthology.orchestra.run.smithi146.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T16:56:15.888 INFO:teuthology.orchestra.run.smithi146.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T16:56:15.891 INFO:teuthology.orchestra.run.smithi146.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T16:56:15.908 INFO:teuthology.orchestra.run.smithi146.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T16:56:15.909 INFO:teuthology.orchestra.run.smithi146.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T16:56:15.911 INFO:teuthology.orchestra.run.smithi146.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T16:56:15.911 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-08-15T16:56:15.911 INFO:teuthology.orchestra.run.smithi146.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T16:56:15.916 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-08-15T16:56:15.941 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-08-15T16:56:16.066 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-08-15T16:56:16.073 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-08-15T16:56:16.096 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-08-15T16:56:16.103 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-08-15T16:56:16.104 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-08-15T16:56:16.110 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-08-15T16:56:16.110 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-08-15T16:56:16.112 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-08-15T16:56:16.112 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-08-15T16:56:16.120 INFO:teuthology.orchestra.run.smithi146.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1375-g2c991c19-1focal [3,557 kB] 2024-08-15T16:56:16.175 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-08-15T16:56:16.176 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-08-15T16:56:16.256 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-08-15T16:56:16.285 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-08-15T16:56:16.286 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-08-15T16:56:16.349 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-08-15T16:56:16.349 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T16:56:16.350 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-08-15T16:56:16.350 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2024-08-15T16:56:16.353 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-08-15T16:56:16.353 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-08-15T16:56:16.356 INFO:teuthology.orchestra.run.smithi161.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-08-15T16:56:16.358 INFO:teuthology.orchestra.run.smithi161.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-08-15T16:56:16.366 INFO:teuthology.orchestra.run.smithi146.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1375-g2c991c19-1focal [761 kB] 2024-08-15T16:56:16.395 INFO:teuthology.orchestra.run.smithi161.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T16:56:16.396 INFO:teuthology.orchestra.run.smithi161.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T16:56:16.398 INFO:teuthology.orchestra.run.smithi161.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T16:56:16.419 INFO:teuthology.orchestra.run.smithi161.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-08-15T16:56:16.424 INFO:teuthology.orchestra.run.smithi146.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1375-g2c991c19-1focal [414 kB] 2024-08-15T16:56:16.426 INFO:teuthology.orchestra.run.smithi161.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-08-15T16:56:16.433 INFO:teuthology.orchestra.run.smithi161.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T16:56:16.466 INFO:teuthology.orchestra.run.smithi146.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1375-g2c991c19-1focal [22.6 kB] 2024-08-15T16:56:16.467 INFO:teuthology.orchestra.run.smithi146.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1375-g2c991c19-1focal [200 kB] 2024-08-15T16:56:16.485 INFO:teuthology.orchestra.run.smithi146.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1375-g2c991c19-1focal [49.1 kB] 2024-08-15T16:56:16.486 INFO:teuthology.orchestra.run.smithi146.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1375-g2c991c19-1focal [390 kB] 2024-08-15T16:56:16.500 INFO:teuthology.orchestra.run.smithi161.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-08-15T16:56:16.501 INFO:teuthology.orchestra.run.smithi161.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-08-15T16:56:16.502 INFO:teuthology.orchestra.run.smithi161.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-08-15T16:56:16.504 INFO:teuthology.orchestra.run.smithi161.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-08-15T16:56:16.509 INFO:teuthology.orchestra.run.smithi161.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-08-15T16:56:16.512 INFO:teuthology.orchestra.run.smithi161.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-08-15T16:56:16.513 INFO:teuthology.orchestra.run.smithi161.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-08-15T16:56:16.516 INFO:teuthology.orchestra.run.smithi161.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-08-15T16:56:16.521 INFO:teuthology.orchestra.run.smithi161.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T16:56:16.526 INFO:teuthology.orchestra.run.smithi146.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1375-g2c991c19-1focal [5,612 kB] 2024-08-15T16:56:16.582 INFO:teuthology.orchestra.run.smithi161.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-08-15T16:56:16.584 INFO:teuthology.orchestra.run.smithi161.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-08-15T16:56:16.598 INFO:teuthology.orchestra.run.smithi161.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-08-15T16:56:16.599 INFO:teuthology.orchestra.run.smithi161.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-08-15T16:56:16.910 INFO:teuthology.orchestra.run.smithi161.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-08-15T16:56:16.953 INFO:teuthology.orchestra.run.smithi161.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-08-15T16:56:16.955 INFO:teuthology.orchestra.run.smithi161.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-08-15T16:56:17.010 INFO:teuthology.orchestra.run.smithi146.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1375-g2c991c19-1focal [116 kB] 2024-08-15T16:56:17.013 INFO:teuthology.orchestra.run.smithi146.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1375-g2c991c19-1focal [479 kB] 2024-08-15T16:56:17.046 INFO:teuthology.orchestra.run.smithi146.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1375-g2c991c19-1focal [26.9 MB] 2024-08-15T16:56:17.067 INFO:teuthology.orchestra.run.smithi161.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-08-15T16:56:17.068 INFO:teuthology.orchestra.run.smithi161.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-08-15T16:56:17.068 INFO:teuthology.orchestra.run.smithi161.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T16:56:17.069 INFO:teuthology.orchestra.run.smithi161.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-08-15T16:56:17.069 INFO:teuthology.orchestra.run.smithi161.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T16:56:17.140 INFO:teuthology.orchestra.run.smithi161.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T16:56:17.140 INFO:teuthology.orchestra.run.smithi161.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-08-15T16:56:17.141 INFO:teuthology.orchestra.run.smithi161.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-08-15T16:56:17.142 INFO:teuthology.orchestra.run.smithi161.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-08-15T16:56:17.143 INFO:teuthology.orchestra.run.smithi161.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-08-15T16:56:17.144 INFO:teuthology.orchestra.run.smithi161.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-08-15T16:56:17.145 INFO:teuthology.orchestra.run.smithi161.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-08-15T16:56:17.154 INFO:teuthology.orchestra.run.smithi161.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-08-15T16:56:17.159 INFO:teuthology.orchestra.run.smithi161.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-08-15T16:56:17.163 INFO:teuthology.orchestra.run.smithi161.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T16:56:17.164 INFO:teuthology.orchestra.run.smithi161.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T16:56:17.165 INFO:teuthology.orchestra.run.smithi161.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T16:56:17.236 INFO:teuthology.orchestra.run.smithi161.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-08-15T16:56:17.239 INFO:teuthology.orchestra.run.smithi161.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-08-15T16:56:17.242 INFO:teuthology.orchestra.run.smithi161.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-08-15T16:56:17.245 INFO:teuthology.orchestra.run.smithi161.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-08-15T16:56:17.248 INFO:teuthology.orchestra.run.smithi161.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-08-15T16:56:17.248 INFO:teuthology.orchestra.run.smithi161.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-08-15T16:56:17.297 INFO:teuthology.orchestra.run.smithi161.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T16:56:17.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2024-08-15T16:56:17.329 INFO:teuthology.orchestra.run.smithi161.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-08-15T16:56:17.479 INFO:teuthology.orchestra.run.smithi161.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T16:56:17.479 INFO:teuthology.orchestra.run.smithi161.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T16:56:17.491 INFO:teuthology.orchestra.run.smithi161.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-08-15T16:56:17.575 INFO:teuthology.orchestra.run.smithi161.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-08-15T16:56:17.575 INFO:teuthology.orchestra.run.smithi161.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-08-15T16:56:17.576 INFO:teuthology.orchestra.run.smithi161.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-08-15T16:56:17.577 INFO:teuthology.orchestra.run.smithi161.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-08-15T16:56:17.579 INFO:teuthology.orchestra.run.smithi161.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-08-15T16:56:17.595 INFO:teuthology.orchestra.run.smithi161.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-08-15T16:56:17.596 INFO:teuthology.orchestra.run.smithi161.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-08-15T16:56:17.596 INFO:teuthology.orchestra.run.smithi161.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-08-15T16:56:17.597 INFO:teuthology.orchestra.run.smithi161.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-08-15T16:56:17.657 INFO:teuthology.orchestra.run.smithi161.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-08-15T16:56:17.658 INFO:teuthology.orchestra.run.smithi161.stdout:Get:107 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-08-15T16:56:17.738 INFO:teuthology.orchestra.run.smithi161.stdout:Get:108 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-08-15T16:56:17.739 INFO:teuthology.orchestra.run.smithi161.stdout:Get:109 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-08-15T16:56:18.253 INFO:teuthology.orchestra.run.smithi146.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1375-g2c991c19-1focal [5,175 kB] 2024-08-15T16:56:18.353 INFO:teuthology.orchestra.run.smithi146.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1375-g2c991c19-1focal [223 kB] 2024-08-15T16:56:18.357 INFO:teuthology.orchestra.run.smithi146.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1375-g2c991c19-1focal [157 kB] 2024-08-15T16:56:18.359 INFO:teuthology.orchestra.run.smithi146.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1375-g2c991c19-1focal [1,594 kB] 2024-08-15T16:56:18.412 INFO:teuthology.orchestra.run.smithi146.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1375-g2c991c19-1focal [4,687 kB] 2024-08-15T16:56:18.591 INFO:teuthology.orchestra.run.smithi146.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1375-g2c991c19-1focal [23.1 MB] 2024-08-15T16:56:19.200 INFO:teuthology.orchestra.run.smithi161.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T16:56:19.205 INFO:teuthology.orchestra.run.smithi161.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T16:56:19.209 INFO:teuthology.orchestra.run.smithi161.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T16:56:19.210 INFO:teuthology.orchestra.run.smithi161.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T16:56:19.211 INFO:teuthology.orchestra.run.smithi161.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T16:56:19.252 INFO:teuthology.orchestra.run.smithi161.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T16:56:19.337 INFO:teuthology.orchestra.run.smithi146.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1375-g2c991c19-1focal [3,944 B] 2024-08-15T16:56:19.338 INFO:teuthology.orchestra.run.smithi146.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1375-g2c991c19-1focal [872 kB] 2024-08-15T16:56:19.350 INFO:teuthology.orchestra.run.smithi146.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1375-g2c991c19-1focal [2,416 kB] 2024-08-15T16:56:19.392 INFO:teuthology.orchestra.run.smithi146.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1375-g2c991c19-1focal [81.4 kB] 2024-08-15T16:56:19.393 INFO:teuthology.orchestra.run.smithi146.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1375-g2c991c19-1focal [120 kB] 2024-08-15T16:56:19.395 INFO:teuthology.orchestra.run.smithi146.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1375-g2c991c19-1focal [6,653 kB] 2024-08-15T16:56:19.644 INFO:teuthology.orchestra.run.smithi146.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1375-g2c991c19-1focal [7,863 kB] 2024-08-15T16:56:19.695 INFO:teuthology.orchestra.run.smithi161.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T16:56:19.775 INFO:teuthology.orchestra.run.smithi146.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1375-g2c991c19-1focal [4,060 B] 2024-08-15T16:56:19.775 INFO:teuthology.orchestra.run.smithi146.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1375-g2c991c19-1focal [53.3 MB] 2024-08-15T16:56:20.341 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 201 MB in 4s (44.8 MB/s) 2024-08-15T16:56:20.471 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T16:56:20.527 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T16:56:20.531 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T16:56:20.585 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T16:56:20.871 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T16:56:20.873 INFO:teuthology.orchestra.run.smithi146.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1375-g2c991c19-1focal [230 kB] 2024-08-15T16:56:20.877 INFO:teuthology.orchestra.run.smithi146.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 17.2.7-1375-g2c991c19-1focal [241 kB] 2024-08-15T16:56:20.880 INFO:teuthology.orchestra.run.smithi146.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 17.2.7-1375-g2c991c19-1focal [22.0 kB] 2024-08-15T16:56:20.881 INFO:teuthology.orchestra.run.smithi146.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 17.2.7-1375-g2c991c19-1focal [13.1 kB] 2024-08-15T16:56:20.881 INFO:teuthology.orchestra.run.smithi146.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1375-g2c991c19-1focal [21.9 kB] 2024-08-15T16:56:20.891 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T16:56:20.927 INFO:teuthology.orchestra.run.smithi146.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1375-g2c991c19-1focal [17.1 MB] 2024-08-15T16:56:20.929 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T16:56:21.216 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T16:56:21.227 INFO:teuthology.orchestra.run.smithi146.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2c991c196a651f8ef0f36c7763ec87a241b25549/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1375-g2c991c19-1focal [80.9 kB] 2024-08-15T16:56:21.232 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T16:56:21.265 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T16:56:21.610 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T16:56:21.630 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T16:56:21.710 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T16:56:21.882 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 201 MB in 6s (33.9 MB/s) 2024-08-15T16:56:22.062 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyperclip. 2024-08-15T16:56:22.075 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:22.121 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141718 files and directories currently installed.) 2024-08-15T16:56:22.125 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-08-15T16:56:22.168 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-08-15T16:56:22.197 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T16:56:22.495 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-wcwidth. 2024-08-15T16:56:22.515 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-08-15T16:56:22.553 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T16:56:22.881 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cmd2. 2024-08-15T16:56:22.897 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-08-15T16:56:22.909 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:22.939 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-08-15T16:56:23.034 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T16:56:23.317 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-08-15T16:56:23.334 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-08-15T16:56:23.418 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T16:56:23.746 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T16:56:23.762 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:23.796 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:23.797 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../004-librbd1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:23.921 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librbd1 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T16:56:24.132 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rados. 2024-08-15T16:56:24.149 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:24.191 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:24.543 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T16:56:24.560 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:24.601 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:24.650 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../005-librados2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:24.809 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librados2 (17.2.7-1375-g2c991c19-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-08-15T16:56:24.879 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T16:56:24.894 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:24.937 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:25.256 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T16:56:25.275 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:25.314 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:25.521 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libcephfs2. 2024-08-15T16:56:25.538 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../006-libcephfs2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:25.579 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:25.634 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T16:56:25.652 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T16:56:25.692 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T16:56:25.923 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rados. 2024-08-15T16:56:25.939 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../007-python3-rados_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:25.970 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T16:56:25.981 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:25.986 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:26.028 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:26.351 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-08-15T16:56:26.371 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../008-python3-ceph-argparse_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:26.389 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T16:56:26.405 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T16:56:26.409 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:26.447 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T16:56:26.678 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cephfs. 2024-08-15T16:56:26.695 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../009-python3-cephfs_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:26.736 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:26.817 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T16:56:26.835 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T16:56:26.875 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T16:56:27.081 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-ceph-common. 2024-08-15T16:56:27.100 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../010-python3-ceph-common_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:27.130 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:27.212 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T16:56:27.228 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T16:56:27.270 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T16:56:27.475 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-prettytable. 2024-08-15T16:56:27.495 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2024-08-15T16:56:27.533 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-08-15T16:56:27.656 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T16:56:27.675 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T16:56:27.714 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T16:56:27.803 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rbd. 2024-08-15T16:56:27.820 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../012-python3-rbd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:27.861 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:28.051 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T16:56:28.068 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T16:56:28.100 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T16:56:28.247 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-08-15T16:56:28.265 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../013-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-08-15T16:56:28.305 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T16:56:28.579 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T16:56:28.596 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:28.635 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-08-15T16:56:28.647 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:28.653 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../014-librabbitmq4_0.10.0-1_amd64.deb ... 2024-08-15T16:56:28.693 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T16:56:29.071 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-08-15T16:56:29.091 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../015-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-08-15T16:56:29.129 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T16:56:29.190 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T16:56:29.206 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:29.248 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:29.491 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-08-15T16:56:29.508 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../016-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-08-15T16:56:29.548 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T16:56:29.578 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T16:56:29.594 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:29.635 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:29.893 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-08-15T16:56:29.909 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../017-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-08-15T16:56:29.951 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T16:56:29.954 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T16:56:29.970 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T16:56:30.012 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T16:56:30.366 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librgw2. 2024-08-15T16:56:30.381 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:30.424 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:30.446 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-08-15T16:56:30.462 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../018-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:30.531 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:31.008 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-08-15T16:56:31.023 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../019-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:31.057 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:31.193 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T16:56:31.211 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:31.251 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:31.427 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-08-15T16:56:31.443 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../020-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-08-15T16:56:31.486 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:56:31.612 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liboath0. 2024-08-15T16:56:31.629 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T16:56:31.670 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T16:56:31.930 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-08-15T16:56:31.948 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../021-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-08-15T16:56:31.981 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T16:56:32.000 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:32.039 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:32.054 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T16:56:32.475 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package librgw2. 2024-08-15T16:56:32.492 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../022-librgw2_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:32.501 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-common. 2024-08-15T16:56:32.517 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:32.533 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:32.559 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:33.277 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rgw. 2024-08-15T16:56:33.296 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../023-python3-rgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:33.335 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:33.705 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liboath0. 2024-08-15T16:56:33.721 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-08-15T16:56:33.763 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-08-15T16:56:34.099 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libradosstriper1. 2024-08-15T16:56:34.117 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../025-libradosstriper1_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:34.182 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:34.610 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-common. 2024-08-15T16:56:34.626 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../026-ceph-common_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:34.660 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:35.148 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-base. 2024-08-15T16:56:35.164 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:35.216 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:36.111 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T16:56:36.126 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T16:56:36.169 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T16:56:36.589 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T16:56:36.605 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T16:56:36.648 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T16:56:36.975 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T16:56:36.995 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T16:56:37.033 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T16:56:37.207 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-base. 2024-08-15T16:56:37.223 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../027-ceph-base_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:37.267 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:37.286 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T16:56:37.304 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:37.344 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:37.716 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T16:56:37.733 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T16:56:37.773 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T16:56:38.118 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T16:56:38.136 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T16:56:38.170 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cryptography. 2024-08-15T16:56:38.176 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T16:56:38.185 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../028-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-08-15T16:56:38.244 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T16:56:38.513 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T16:56:38.529 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T16:56:38.571 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T16:56:38.623 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-openssl. 2024-08-15T16:56:38.638 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../029-python3-openssl_19.0.0-1build1_all.deb ... 2024-08-15T16:56:38.681 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-08-15T16:56:38.899 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T16:56:38.916 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T16:56:38.957 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T16:56:39.025 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-packaging. 2024-08-15T16:56:39.041 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../030-python3-packaging_20.3-1_all.deb ... 2024-08-15T16:56:39.083 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-packaging (20.3-1) ... 2024-08-15T16:56:39.252 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T16:56:39.271 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T16:56:39.310 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T16:56:39.360 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-08-15T16:56:39.380 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../031-ceph-mgr-modules-core_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:39.428 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:39.554 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T16:56:39.574 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T16:56:39.604 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T16:56:39.841 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-bcrypt. 2024-08-15T16:56:39.857 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../032-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-08-15T16:56:39.902 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T16:56:39.907 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-mako. 2024-08-15T16:56:39.924 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T16:56:39.966 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T16:56:40.210 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cherrypy3. 2024-08-15T16:56:40.228 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../033-python3-cherrypy3_8.9.1-6_all.deb ... 2024-08-15T16:56:40.268 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-08-15T16:56:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T16:56:40.312 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T16:56:40.351 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T16:56:40.638 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-natsort. 2024-08-15T16:56:40.654 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../034-python3-natsort_7.0.1-1_all.deb ... 2024-08-15T16:56:40.688 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T16:56:40.688 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-08-15T16:56:40.705 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T16:56:40.746 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T16:56:41.023 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webob. 2024-08-15T16:56:41.043 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T16:56:41.049 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-jwt. 2024-08-15T16:56:41.068 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../035-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-08-15T16:56:41.090 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T16:56:41.107 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T16:56:41.443 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T16:56:41.452 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-logutils. 2024-08-15T16:56:41.458 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T16:56:41.469 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-7_all.deb ... 2024-08-15T16:56:41.501 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T16:56:41.511 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-08-15T16:56:41.804 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T16:56:41.820 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T16:56:41.838 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-markupsafe. 2024-08-15T16:56:41.854 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../037-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-08-15T16:56:41.862 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T16:56:41.896 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-08-15T16:56:42.233 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-mako. 2024-08-15T16:56:42.248 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T16:56:42.250 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../038-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-08-15T16:56:42.265 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T16:56:42.282 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T16:56:42.309 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T16:56:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T16:56:42.652 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-simplegeneric. 2024-08-15T16:56:42.660 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T16:56:42.669 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2024-08-15T16:56:42.701 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T16:56:42.710 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-08-15T16:56:43.012 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-paste. 2024-08-15T16:56:43.031 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T16:56:43.038 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-singledispatch. 2024-08-15T16:56:43.054 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-08-15T16:56:43.070 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T16:56:43.122 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T16:56:43.353 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T16:56:43.372 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T16:56:43.402 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T16:56:43.424 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webob. 2024-08-15T16:56:43.441 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.5-2_all.deb ... 2024-08-15T16:56:43.507 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2024-08-15T16:56:43.765 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T16:56:43.781 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T16:56:43.822 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T16:56:43.835 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-soupsieve. 2024-08-15T16:56:43.855 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../042-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-08-15T16:56:43.918 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T16:56:44.126 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T16:56:44.142 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T16:56:44.171 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-bs4. 2024-08-15T16:56:44.184 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T16:56:44.190 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../043-python3-bs4_4.8.2-1_all.deb ... 2024-08-15T16:56:44.254 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-08-15T16:56:44.495 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T16:56:44.512 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T16:56:44.578 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T16:56:44.682 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-waitress. 2024-08-15T16:56:44.698 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../044-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-08-15T16:56:44.744 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T16:56:44.941 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T16:56:44.958 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T16:56:44.999 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T16:56:45.081 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-tempita. 2024-08-15T16:56:45.100 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../045-python3-tempita_0.5.2-6_all.deb ... 2024-08-15T16:56:45.139 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-08-15T16:56:45.378 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T16:56:45.394 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:45.436 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:45.468 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-paste. 2024-08-15T16:56:45.485 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../046-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-08-15T16:56:45.526 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T16:56:45.739 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T16:56:45.758 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:45.797 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:45.854 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-08-15T16:56:45.870 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../047-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-08-15T16:56:45.912 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T16:56:46.240 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pastedeploy. 2024-08-15T16:56:46.257 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../048-python3-pastedeploy_2.1.0-1_all.deb ... 2024-08-15T16:56:46.298 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-08-15T16:56:46.325 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T16:56:46.341 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:46.385 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:46.626 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webtest. 2024-08-15T16:56:46.642 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../049-python3-webtest_2.0.32-2_all.deb ... 2024-08-15T16:56:46.684 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-08-15T16:56:46.987 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pecan. 2024-08-15T16:56:47.005 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../050-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-08-15T16:56:47.045 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T16:56:47.240 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T16:56:47.259 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:47.298 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:47.450 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-werkzeug. 2024-08-15T16:56:47.466 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../051-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-08-15T16:56:47.733 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T16:56:48.662 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-08-15T16:56:48.678 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../052-libsqlite3-mod-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:48.720 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:49.102 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr. 2024-08-15T16:56:49.118 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../053-ceph-mgr_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:49.178 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:49.705 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mon. 2024-08-15T16:56:49.721 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../054-ceph-mon_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:49.763 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:49.852 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph. 2024-08-15T16:56:49.867 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:49.910 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:50.261 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T16:56:50.277 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:50.322 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:50.581 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-osd. 2024-08-15T16:56:50.600 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../055-ceph-osd_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:50.639 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:50.747 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T16:56:50.763 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:50.805 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:51.341 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephadm. 2024-08-15T16:56:51.358 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:51.408 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:51.703 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T16:56:51.719 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T16:56:51.761 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T16:56:52.106 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T16:56:52.121 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:52.163 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:52.492 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T16:56:52.511 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T16:56:52.550 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T16:56:52.836 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T16:56:52.842 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph. 2024-08-15T16:56:52.851 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T16:56:52.858 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../056-ceph_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:52.894 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T16:56:52.900 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:53.163 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T16:56:53.180 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T16:56:53.221 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T16:56:53.245 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-fuse. 2024-08-15T16:56:53.262 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../057-ceph-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:53.328 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:53.532 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-routes. 2024-08-15T16:56:53.549 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T16:56:53.590 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T16:56:53.740 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mds. 2024-08-15T16:56:53.760 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../058-ceph-mds_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:53.798 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:53.926 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T16:56:53.944 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:53.984 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:54.376 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephadm. 2024-08-15T16:56:54.392 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../059-cephadm_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:56:54.434 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:54.779 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-jinja2. 2024-08-15T16:56:54.795 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../060-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-08-15T16:56:54.862 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T16:56:55.232 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-08-15T16:56:55.247 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../061-ceph-mgr-cephadm_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:55.290 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:55.373 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T16:56:55.390 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T16:56:55.431 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T16:56:55.626 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-repoze.lru. 2024-08-15T16:56:55.645 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../062-python3-repoze.lru_0.7-2_all.deb ... 2024-08-15T16:56:55.684 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-08-15T16:56:55.742 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T16:56:55.758 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T16:56:55.800 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T16:56:55.937 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-underscore. 2024-08-15T16:56:55.954 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../063-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-08-15T16:56:56.012 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T16:56:56.128 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T16:56:56.145 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T16:56:56.186 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T16:56:56.298 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-08-15T16:56:56.314 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../064-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-08-15T16:56:56.381 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T16:56:56.725 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-routes. 2024-08-15T16:56:56.744 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../065-python3-routes_2.4.1-2_all.deb ... 2024-08-15T16:56:56.783 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-08-15T16:56:57.145 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-08-15T16:56:57.162 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../066-ceph-mgr-dashboard_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:57.203 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:57.540 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T16:56:57.557 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T16:56:57.597 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T16:56:58.118 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T16:56:58.134 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T16:56:58.176 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T16:56:58.558 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-decorator. 2024-08-15T16:56:58.575 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../067-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-08-15T16:56:58.579 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T16:56:58.595 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T16:56:58.624 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T16:56:58.637 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T16:56:58.952 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-08-15T16:56:58.972 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../068-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-08-15T16:56:59.010 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T16:56:59.329 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T16:56:59.346 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:56:59.363 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-scipy. 2024-08-15T16:56:59.381 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../069-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-08-15T16:56:59.386 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:56:59.421 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-08-15T16:57:00.498 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T16:57:00.514 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T16:57:00.556 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T16:57:00.790 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-08-15T16:57:00.809 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-08-15T16:57:00.839 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:00.892 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T16:57:00.910 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T16:57:00.950 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T16:57:01.287 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T16:57:01.303 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T16:57:01.345 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T16:57:01.376 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-joblib. 2024-08-15T16:57:01.396 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../071-python3-joblib_0.14.0-3_all.deb ... 2024-08-15T16:57:01.434 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-08-15T16:57:01.656 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T16:57:01.676 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T16:57:01.714 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T16:57:01.821 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-sklearn. 2024-08-15T16:57:01.837 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../072-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-08-15T16:57:01.904 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:02.001 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T16:57:02.017 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T16:57:02.050 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T16:57:02.404 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T16:57:02.420 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T16:57:02.461 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T16:57:02.587 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-08-15T16:57:02.603 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../073-ceph-mgr-diskprediction-local_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:02.662 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:02.748 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T16:57:02.768 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T16:57:02.806 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T16:57:03.236 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T16:57:03.253 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T16:57:03.294 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T16:57:03.623 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T16:57:03.639 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T16:57:03.673 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T16:57:04.018 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T16:57:04.037 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T16:57:04.076 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T16:57:04.115 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-cachetools. 2024-08-15T16:57:04.131 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../074-python3-cachetools_4.0.0-1_all.deb ... 2024-08-15T16:57:04.222 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-08-15T16:57:04.492 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyasn1. 2024-08-15T16:57:04.512 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../075-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-08-15T16:57:04.550 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T16:57:04.641 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T16:57:04.657 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:04.698 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:04.921 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-08-15T16:57:04.938 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../076-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-08-15T16:57:05.003 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T16:57:05.315 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-rsa. 2024-08-15T16:57:05.330 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../077-python3-rsa_4.0-3_all.deb ... 2024-08-15T16:57:05.376 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T16:57:05.393 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T16:57:05.406 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-rsa (4.0-3) ... 2024-08-15T16:57:05.434 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T16:57:05.701 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-google-auth. 2024-08-15T16:57:05.720 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../078-python3-google-auth_1.5.1-2_all.deb ... 2024-08-15T16:57:05.783 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-08-15T16:57:05.804 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T16:57:05.822 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:57:05.862 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:06.104 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-blinker. 2024-08-15T16:57:06.124 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../079-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-08-15T16:57:06.127 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package jq. 2024-08-15T16:57:06.146 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:57:06.178 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T16:57:06.185 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:06.455 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package socat. 2024-08-15T16:57:06.471 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T16:57:06.506 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-oauthlib. 2024-08-15T16:57:06.512 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T16:57:06.525 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../080-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-08-15T16:57:06.565 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T16:57:06.841 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T16:57:06.858 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T16:57:06.899 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T16:57:07.004 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-08-15T16:57:07.020 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../081-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-08-15T16:57:07.062 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T16:57:07.262 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-test. 2024-08-15T16:57:07.279 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:07.319 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:07.399 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-websocket. 2024-08-15T16:57:07.414 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../082-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-08-15T16:57:07.456 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T16:57:07.811 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-kubernetes. 2024-08-15T16:57:07.827 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../083-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-08-15T16:57:07.869 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T16:57:08.459 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-08-15T16:57:08.478 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../084-ceph-mgr-k8sevents_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:08.517 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:08.874 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libonig5:amd64. 2024-08-15T16:57:08.891 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../085-libonig5_6.9.4-1_amd64.deb ... 2024-08-15T16:57:08.932 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-08-15T16:57:09.310 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjq1:amd64. 2024-08-15T16:57:09.328 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../086-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:57:09.368 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:09.670 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package jq. 2024-08-15T16:57:09.689 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../087-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:57:09.745 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:10.015 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package socat. 2024-08-15T16:57:10.034 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../088-socat_1.7.3.3-2_amd64.deb ... 2024-08-15T16:57:10.073 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking socat (1.7.3.3-2) ... 2024-08-15T16:57:10.410 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package xmlstarlet. 2024-08-15T16:57:10.427 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../089-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-08-15T16:57:10.468 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-08-15T16:57:10.831 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-test. 2024-08-15T16:57:10.849 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../090-ceph-test_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:10.889 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:11.916 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T16:57:11.933 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:11.974 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:12.429 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T16:57:12.446 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:12.487 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:12.826 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T16:57:12.842 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T16:57:12.909 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T16:57:13.154 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T16:57:13.172 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:13.212 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:13.464 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T16:57:13.481 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:13.514 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:13.777 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T16:57:13.794 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T16:57:13.835 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:14.381 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T16:57:14.397 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:14.438 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:14.866 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T16:57:14.883 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T16:57:14.942 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T16:57:15.263 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T16:57:15.282 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T16:57:15.354 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:15.371 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ceph-volume. 2024-08-15T16:57:15.388 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../091-ceph-volume_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:15.454 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:15.884 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-mirror. 2024-08-15T16:57:15.902 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../092-cephfs-mirror_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:15.942 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:16.306 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-colorama. 2024-08-15T16:57:16.319 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../093-python3-colorama_0.4.3-1build1_all.deb ... 2024-08-15T16:57:16.361 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-08-15T16:57:16.649 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-shell. 2024-08-15T16:57:16.666 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../094-cephfs-shell_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:16.707 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:16.985 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package cephfs-top. 2024-08-15T16:57:17.002 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../095-cephfs-top_17.2.7-1375-g2c991c19-1focal_all.deb ... 2024-08-15T16:57:17.037 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T16:57:17.043 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:17.055 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T16:57:17.095 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T16:57:17.296 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package fonts-mathjax. 2024-08-15T16:57:17.315 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T16:57:17.353 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:17.431 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T16:57:17.448 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T16:57:17.489 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T16:57:17.816 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-attr. 2024-08-15T16:57:17.836 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T16:57:17.874 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libcephfs-dev. 2024-08-15T16:57:17.883 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T16:57:17.891 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../097-libcephfs-dev_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:17.932 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:18.220 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T16:57:18.236 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T16:57:18.277 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T16:57:18.343 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package nvme-cli. 2024-08-15T16:57:18.359 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2024-08-15T16:57:18.419 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T16:57:18.564 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T16:57:18.582 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T16:57:18.647 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T16:57:18.740 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libjs-mathjax. 2024-08-15T16:57:18.756 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../099-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-08-15T16:57:18.822 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:18.975 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T16:57:18.993 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T16:57:19.033 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T16:57:19.528 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T16:57:19.544 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T16:57:19.577 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T16:57:19.864 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T16:57:19.884 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T16:57:19.922 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T16:57:20.258 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T16:57:20.275 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T16:57:20.316 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T16:57:20.473 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python-natsort-doc. 2024-08-15T16:57:20.494 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../100-python-natsort-doc_7.0.1-1_all.deb ... 2024-08-15T16:57:20.539 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-08-15T16:57:20.628 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-py. 2024-08-15T16:57:20.648 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T16:57:20.685 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T16:57:20.900 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-atomicwrites. 2024-08-15T16:57:20.921 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../101-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-08-15T16:57:20.958 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T16:57:21.047 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T16:57:21.067 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T16:57:21.105 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T16:57:21.269 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-attr. 2024-08-15T16:57:21.286 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../102-python3-attr_19.3.0-2_all.deb ... 2024-08-15T16:57:21.319 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-08-15T16:57:21.416 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T16:57:21.434 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T16:57:21.474 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T16:57:21.655 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-webencodings. 2024-08-15T16:57:21.672 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../103-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-08-15T16:57:21.713 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T16:57:21.811 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T16:57:21.830 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T16:57:21.886 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T16:57:22.032 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-html5lib. 2024-08-15T16:57:22.053 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../104-python3-html5lib_1.0.1-2_all.deb ... 2024-08-15T16:57:22.091 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-08-15T16:57:22.163 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T16:57:22.183 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T16:57:22.222 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T16:57:22.452 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-08-15T16:57:22.469 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../105-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-08-15T16:57:22.535 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T16:57:22.959 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package radosgw. 2024-08-15T16:57:22.978 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:22.996 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pastescript. 2024-08-15T16:57:23.017 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-08-15T16:57:23.018 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:23.055 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-08-15T16:57:23.416 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pluggy. 2024-08-15T16:57:23.435 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-2_all.deb ... 2024-08-15T16:57:23.475 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-08-15T16:57:23.819 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-psutil. 2024-08-15T16:57:23.836 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../108-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-08-15T16:57:23.899 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T16:57:24.222 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-py. 2024-08-15T16:57:24.239 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../109-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-08-15T16:57:24.296 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T16:57:24.667 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pyinotify. 2024-08-15T16:57:24.686 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-08-15T16:57:24.724 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T16:57:24.750 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T16:57:24.767 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:24.808 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:25.061 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-pytest. 2024-08-15T16:57:25.079 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../111-python3-pytest_4.6.9-1_all.deb ... 2024-08-15T16:57:25.118 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-08-15T16:57:25.129 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T16:57:25.438 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T16:57:25.488 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package python3-simplejson. 2024-08-15T16:57:25.509 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../112-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-08-15T16:57:25.547 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T16:57:25.547 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T16:57:25.690 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T16:57:25.861 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-08-15T16:57:25.879 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../113-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-08-15T16:57:25.891 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:25.919 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T16:57:26.479 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user cephadm....done 2024-08-15T16:57:26.633 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:26.723 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package radosgw. 2024-08-15T16:57:26.739 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../114-radosgw_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:26.783 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T16:57:26.805 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:27.127 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T16:57:27.428 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:27.570 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T16:57:27.688 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T16:57:27.998 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T16:57:28.426 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T16:57:28.525 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package rbd-fuse. 2024-08-15T16:57:28.543 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../115-rbd-fuse_17.2.7-1375-g2c991c19-1focal_amd64.deb ... 2024-08-15T16:57:28.583 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:28.727 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T16:57:28.926 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-attr (19.3.0-2) ... 2024-08-15T16:57:29.020 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T16:57:29.221 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-08-15T16:57:29.313 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T16:57:29.348 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-08-15T16:57:29.509 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2024-08-15T16:57:29.564 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T16:57:29.702 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:29.848 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:30.174 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:30.300 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T16:57:30.322 INFO:teuthology.orchestra.run.smithi146.stdout:Adding system user cephadm....done 2024-08-15T16:57:30.483 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:30.593 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T16:57:30.643 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-08-15T16:57:30.885 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T16:57:30.995 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-08-15T16:57:31.187 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T16:57:31.313 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T16:57:31.313 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-08-15T16:57:31.456 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-08-15T16:57:31.582 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-08-15T16:57:31.697 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T16:57:31.967 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-08-15T16:57:31.998 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T16:57:32.307 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T16:57:32.419 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-08-15T16:57:32.650 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T16:57:32.737 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-08-15T16:57:32.943 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T16:57:33.063 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-08-15T16:57:33.277 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T16:57:33.355 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-08-15T16:57:33.578 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T16:57:33.640 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-08-15T16:57:33.887 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T16:57:33.966 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-ceph-argparse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:34.030 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T16:57:34.147 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T16:57:34.259 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:34.418 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-08-15T16:57:34.440 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T16:57:34.858 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T16:57:35.168 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T16:57:35.180 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-08-15T16:57:35.495 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-08-15T16:57:35.536 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T16:57:35.779 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-08-15T16:57:35.930 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-08-15T16:57:36.104 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T16:57:36.306 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-08-15T16:57:36.605 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:36.608 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-08-15T16:57:36.867 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-08-15T16:57:36.914 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:37.049 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T16:57:37.193 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-08-15T16:57:37.409 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T16:57:37.494 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-08-15T16:57:37.534 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T16:57:37.660 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T16:57:37.829 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-08-15T16:57:37.953 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T16:57:38.122 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-packaging (20.3-1) ... 2024-08-15T16:57:38.279 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:38.397 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T16:57:38.414 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-08-15T16:57:38.514 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T16:57:38.540 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-08-15T16:57:38.666 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-08-15T16:57:38.841 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T16:57:38.943 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-08-15T16:57:39.150 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T16:57:39.311 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-08-15T16:57:39.451 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T16:57:39.577 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T16:57:39.612 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-08-15T16:57:39.887 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T16:57:39.980 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-08-15T16:57:40.188 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T16:57:40.348 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up socat (1.7.3.3-2) ... 2024-08-15T16:57:40.483 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:40.505 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T16:57:40.826 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:40.911 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T16:57:40.969 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-08-15T16:57:41.158 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T16:57:41.301 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T16:57:41.303 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-08-15T16:57:41.429 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-08-15T16:57:41.555 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-08-15T16:57:41.627 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T16:57:41.873 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-08-15T16:57:41.994 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T16:57:42.137 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T16:57:42.226 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:42.263 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T16:57:42.361 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-08-15T16:57:42.487 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-08-15T16:57:42.581 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T16:57:42.813 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-08-15T16:57:42.857 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:43.016 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T16:57:43.106 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-08-15T16:57:43.440 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-08-15T16:57:43.566 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-08-15T16:57:43.851 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-08-15T16:57:44.119 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-08-15T16:57:44.436 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-08-15T16:57:44.510 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:44.614 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:44.748 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T16:57:44.856 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-08-15T16:57:45.081 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-08-15T16:57:45.125 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T16:57:45.207 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-08-15T16:57:45.426 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T16:57:45.591 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-08-15T16:57:45.735 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:45.861 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T16:57:45.993 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-08-15T16:57:46.119 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-08-15T16:57:46.221 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:46.228 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webob (1:1.8.5-2) ... 2024-08-15T16:57:46.346 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T16:57:46.571 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-08-15T16:57:46.673 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:46.799 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T16:57:46.880 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librados2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:47.006 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-08-15T16:57:47.125 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:48.147 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:48.273 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T16:57:48.399 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T16:57:48.478 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:48.525 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:48.646 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:48.676 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T16:57:48.772 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-08-15T16:57:48.977 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T16:57:49.103 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T16:57:49.157 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-08-15T16:57:49.458 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-08-15T16:57:49.489 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:49.615 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:49.767 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-08-15T16:57:49.835 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T16:57:49.836 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T16:57:49.893 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-08-15T16:57:50.236 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libcephfs2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:50.267 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:50.362 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-08-15T16:57:50.392 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T16:57:50.694 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T16:57:50.739 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libradosstriper1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:50.890 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-08-15T16:57:51.037 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T16:57:51.241 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-08-15T16:57:51.364 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:51.490 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:51.616 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:51.742 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:51.910 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:52.036 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T16:57:52.296 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librbd1 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:52.345 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T16:57:52.422 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-08-15T16:57:52.548 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-08-15T16:57:52.663 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:52.699 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-08-15T16:57:52.764 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:52.825 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rsa (4.0-3) ... 2024-08-15T16:57:52.915 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:53.041 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T16:57:53.134 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-08-15T16:57:53.260 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-08-15T16:57:53.696 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-modules-core (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:53.823 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:53.978 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:54.046 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T16:57:54.046 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-08-15T16:57:54.480 INFO:teuthology.orchestra.run.smithi161.stdout:Adding group ceph....done 2024-08-15T16:57:54.507 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libcephfs-dev (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:54.625 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-08-15T16:57:54.898 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user ceph....done 2024-08-15T16:57:54.919 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-08-15T16:57:55.058 INFO:teuthology.orchestra.run.smithi161.stdout:Setting system user ceph properties....done 2024-08-15T16:57:55.092 INFO:teuthology.orchestra.run.smithi161.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T16:57:55.228 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-08-15T16:57:55.235 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T16:57:55.557 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T16:57:55.614 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rados (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:55.740 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up librgw2 (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:55.891 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-top (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.008 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rbd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.012 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.143 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up rbd-fuse (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.218 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T16:57:56.218 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T16:57:56.302 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-routes (2.4.1-2) ... 2024-08-15T16:57:56.628 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-08-15T16:57:56.699 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.831 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:56.930 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-rgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:57.081 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-cephfs (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:57.155 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T16:57:57.155 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T16:57:57.207 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-shell (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:57.333 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-08-15T16:57:57.817 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:58.203 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T16:57:58.277 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-common (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:58.651 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:58.846 INFO:teuthology.orchestra.run.smithi146.stdout:Adding group ceph....done 2024-08-15T16:57:58.875 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T16:57:58.875 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T16:57:59.198 INFO:teuthology.orchestra.run.smithi146.stdout:Adding system user ceph....done 2024-08-15T16:57:59.328 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:57:59.408 INFO:teuthology.orchestra.run.smithi146.stdout:Setting system user ceph properties....done 2024-08-15T16:57:59.433 INFO:teuthology.orchestra.run.smithi146.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-08-15T16:57:59.543 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T16:57:59.543 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T16:57:59.584 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-08-15T16:57:59.894 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-08-15T16:57:59.987 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:00.320 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up cephfs-mirror (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:00.474 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T16:58:00.474 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T16:58:00.549 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T16:58:00.549 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-08-15T16:58:00.914 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:00.955 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-test (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.040 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.106 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up radosgw (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.183 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.379 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T16:58:01.379 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-08-15T16:58:01.403 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T16:58:01.403 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T16:58:01.834 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.976 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:01.990 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-base (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:02.398 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-08-15T16:58:02.420 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:02.554 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:02.681 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T16:58:02.841 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T16:58:02.859 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mds (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:03.096 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T16:58:03.096 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-08-15T16:58:03.117 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T16:58:03.552 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:03.780 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T16:58:03.780 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-08-15T16:58:04.195 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-osd (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:04.708 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T16:58:04.708 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-08-15T16:58:05.138 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:05.264 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:05.432 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mon (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:05.645 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T16:58:05.645 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-08-15T16:58:05.995 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T16:58:06.306 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-cephadm (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:06.444 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:06.570 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-mgr-dashboard (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:06.637 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T16:58:06.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T16:58:06.716 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2024-08-15T16:58:06.754 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ceph-volume (17.2.7-1375-g2c991c19-1focal) ... 2024-08-15T16:58:06.881 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T16:58:06.933 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2024-08-15T16:58:06.934 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2024-08-15T16:58:07.066 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T16:58:07.155 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T16:58:07.156 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T16:58:07.156 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T16:58:07.156 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2024-08-15T16:58:07.156 INFO:teuthology.orchestra.run.smithi161.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T16:58:07.156 INFO:teuthology.orchestra.run.smithi161.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T16:58:07.157 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T16:58:07.157 INFO:teuthology.orchestra.run.smithi161.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T16:58:07.158 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2024-08-15T16:58:07.158 INFO:teuthology.orchestra.run.smithi161.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T16:58:07.159 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T16:58:07.210 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2024-08-15T16:58:07.211 INFO:teuthology.orchestra.run.smithi161.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T16:58:07.211 INFO:teuthology.orchestra.run.smithi161.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T16:58:07.211 INFO:teuthology.orchestra.run.smithi161.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T16:58:07.212 INFO:teuthology.orchestra.run.smithi161.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T16:58:07.213 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2024-08-15T16:58:07.214 INFO:teuthology.orchestra.run.smithi161.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T16:58:07.271 INFO:teuthology.orchestra.run.smithi161.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T16:58:07.271 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 10.4 MB of archives. 2024-08-15T16:58:07.271 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T16:58:07.271 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T16:58:07.367 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T16:58:07.408 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T16:58:07.410 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T16:58:07.411 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T16:58:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T16:58:07.426 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T16:58:07.447 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T16:58:07.482 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T16:58:07.507 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T16:58:07.517 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T16:58:07.523 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T16:58:07.550 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T16:58:07.588 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T16:58:07.604 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T16:58:07.612 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T16:58:07.631 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T16:58:07.632 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T16:58:07.640 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T16:58:07.646 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T16:58:07.646 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T16:58:07.661 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T16:58:07.661 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T16:58:07.668 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T16:58:07.678 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T16:58:07.693 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T16:58:07.696 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T16:58:07.697 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T16:58:07.698 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T16:58:07.698 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T16:58:08.016 INFO:teuthology.orchestra.run.smithi161.stdout:Preconfiguring packages ... 2024-08-15T16:58:08.150 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 10.4 MB in 0s (21.9 MB/s) 2024-08-15T16:58:08.429 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:08.434 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:08.597 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:09.766 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:10.303 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-08-15T16:58:10.892 INFO:teuthology.orchestra.run.smithi146.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T16:58:10.895 DEBUG:teuthology.orchestra.run.smithi146:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2024-08-15T16:58:10.965 INFO:teuthology.orchestra.run.smithi146.stdout:Reading package lists... 2024-08-15T16:58:11.123 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:11.128 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:11.184 INFO:teuthology.orchestra.run.smithi146.stdout:Building dependency tree... 2024-08-15T16:58:11.184 INFO:teuthology.orchestra.run.smithi146.stdout:Reading state information... 2024-08-15T16:58:11.285 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:11.412 INFO:teuthology.orchestra.run.smithi146.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-08-15T16:58:11.412 INFO:teuthology.orchestra.run.smithi146.stdout:flex is already the newest version (2.6.4-6.2). 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages were automatically installed and are no longer required: 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-08-15T16:58:11.413 INFO:teuthology.orchestra.run.smithi146.stdout:Use 'sudo apt autoremove' to remove them. 2024-08-15T16:58:11.414 INFO:teuthology.orchestra.run.smithi146.stdout:The following additional packages will be installed: 2024-08-15T16:58:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-08-15T16:58:11.414 INFO:teuthology.orchestra.run.smithi146.stdout: libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy libssl1.1 libteamdctl0 2024-08-15T16:58:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid1 modemmanager network-manager-pptp ppp pptp-linux usb-modeswitch 2024-08-15T16:58:11.415 INFO:teuthology.orchestra.run.smithi146.stdout: usb-modeswitch-data wpasupplicant 2024-08-15T16:58:11.416 INFO:teuthology.orchestra.run.smithi146.stdout:Suggested packages: 2024-08-15T16:58:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils comgt wvdial 2024-08-15T16:58:11.416 INFO:teuthology.orchestra.run.smithi146.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-08-15T16:58:11.467 INFO:teuthology.orchestra.run.smithi146.stdout:The following NEW packages will be installed: 2024-08-15T16:58:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-08-15T16:58:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libqmi-glib5 libqmi-proxy 2024-08-15T16:58:11.468 INFO:teuthology.orchestra.run.smithi146.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-08-15T16:58:11.469 INFO:teuthology.orchestra.run.smithi146.stdout: ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2024-08-15T16:58:11.470 INFO:teuthology.orchestra.run.smithi146.stdout:The following packages will be upgraded: 2024-08-15T16:58:11.471 INFO:teuthology.orchestra.run.smithi146.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-08-15T16:58:11.525 INFO:teuthology.orchestra.run.smithi146.stdout:6 upgraded, 23 newly installed, 0 to remove and 288 not upgraded. 2024-08-15T16:58:11.525 INFO:teuthology.orchestra.run.smithi146.stdout:Need to get 10.4 MB of archives. 2024-08-15T16:58:11.525 INFO:teuthology.orchestra.run.smithi146.stdout:After this operation, 33.5 MB of additional disk space will be used. 2024-08-15T16:58:11.525 INFO:teuthology.orchestra.run.smithi146.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-08-15T16:58:11.661 INFO:teuthology.orchestra.run.smithi146.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-08-15T16:58:11.662 INFO:teuthology.orchestra.run.smithi146.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-08-15T16:58:11.662 INFO:teuthology.orchestra.run.smithi146.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-08-15T16:58:11.668 INFO:teuthology.orchestra.run.smithi146.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-08-15T16:58:11.669 INFO:teuthology.orchestra.run.smithi146.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-08-15T16:58:11.684 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:11.685 INFO:teuthology.orchestra.run.smithi146.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-08-15T16:58:11.686 INFO:teuthology.orchestra.run.smithi146.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-08-15T16:58:11.687 INFO:teuthology.orchestra.run.smithi146.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-08-15T16:58:11.688 INFO:teuthology.orchestra.run.smithi146.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-08-15T16:58:11.689 INFO:teuthology.orchestra.run.smithi146.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-08-15T16:58:11.695 INFO:teuthology.orchestra.run.smithi146.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-08-15T16:58:11.695 INFO:teuthology.orchestra.run.smithi146.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-08-15T16:58:11.697 INFO:teuthology.orchestra.run.smithi146.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-08-15T16:58:11.697 INFO:teuthology.orchestra.run.smithi146.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-08-15T16:58:11.711 INFO:teuthology.orchestra.run.smithi146.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-08-15T16:58:11.712 INFO:teuthology.orchestra.run.smithi146.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-08-15T16:58:11.718 INFO:teuthology.orchestra.run.smithi146.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-08-15T16:58:11.726 INFO:teuthology.orchestra.run.smithi146.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-08-15T16:58:11.726 INFO:teuthology.orchestra.run.smithi146.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-08-15T16:58:11.740 INFO:teuthology.orchestra.run.smithi146.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-08-15T16:58:11.740 INFO:teuthology.orchestra.run.smithi146.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-08-15T16:58:11.747 INFO:teuthology.orchestra.run.smithi146.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-08-15T16:58:11.757 INFO:teuthology.orchestra.run.smithi146.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-08-15T16:58:11.773 INFO:teuthology.orchestra.run.smithi146.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-08-15T16:58:11.776 INFO:teuthology.orchestra.run.smithi146.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-08-15T16:58:11.776 INFO:teuthology.orchestra.run.smithi146.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-08-15T16:58:11.777 INFO:teuthology.orchestra.run.smithi146.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-08-15T16:58:11.777 INFO:teuthology.orchestra.run.smithi146.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-08-15T16:58:11.874 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:12.098 INFO:teuthology.orchestra.run.smithi146.stdout:Preconfiguring packages ... 2024-08-15T16:58:12.222 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:12.246 INFO:teuthology.orchestra.run.smithi146.stdout:Fetched 10.4 MB in 0s (35.0 MB/s) 2024-08-15T16:58:12.547 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:12.552 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:12.661 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:12.666 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:12.714 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:12.993 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T16:58:13.531 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:13.662 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T16:58:13.830 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:14.095 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T16:58:14.208 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T16:58:14.737 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T16:58:14.755 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T16:58:14.795 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T16:58:15.123 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package dump. 2024-08-15T16:58:15.143 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T16:58:15.181 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T16:58:15.230 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:15.235 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:15.360 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:15.493 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package indent. 2024-08-15T16:58:15.514 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:58:15.551 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T16:58:15.794 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-08-15T16:58:15.879 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T16:58:15.897 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T16:58:15.937 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T16:58:15.974 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-08-15T16:58:16.293 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T16:58:16.311 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T16:58:16.330 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:16.351 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T16:58:16.620 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T16:58:16.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:16.678 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:16.991 INFO:teuthology.orchestra.run.smithi146.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 152983 files and directories currently installed.) 2024-08-15T16:58:16.996 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:17.056 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T16:58:17.073 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:17.115 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:17.124 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T16:58:17.376 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T16:58:17.395 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:17.426 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:17.612 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:17.745 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T16:58:17.766 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:17.779 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-08-15T16:58:17.820 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:18.268 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T16:58:18.393 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T16:58:18.411 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-08-15T16:58:18.412 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T16:58:18.615 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T16:58:18.929 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-08-15T16:58:18.947 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-08-15T16:58:18.987 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-08-15T16:58:19.357 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package dump. 2024-08-15T16:58:19.374 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-08-15T16:58:19.415 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking dump (0.4b46-6) ... 2024-08-15T16:58:19.760 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package indent. 2024-08-15T16:58:19.779 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-08-15T16:58:19.818 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T16:58:20.113 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T16:58:20.134 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T16:58:20.163 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-08-15T16:58:20.181 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-08-15T16:58:20.221 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T16:58:20.346 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:20.574 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-08-15T16:58:20.594 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-08-15T16:58:20.641 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T16:58:20.791 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T16:58:20.808 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:20.849 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:20.919 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-08-15T16:58:20.937 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-08-15T16:58:21.002 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:21.202 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T16:58:21.222 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:21.260 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:21.355 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-08-15T16:58:21.374 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:21.413 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:21.512 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T16:58:21.531 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T16:58:21.571 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:21.809 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmbim-proxy. 2024-08-15T16:58:21.830 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:22.033 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:22.083 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T16:58:22.101 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T16:58:22.141 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T16:58:22.519 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package modemmanager. 2024-08-15T16:58:22.536 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:22.588 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:23.138 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-08-15T16:58:23.155 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:23.196 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:23.234 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T16:58:23.252 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T16:58:23.292 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T16:58:23.590 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libndp0:amd64. 2024-08-15T16:58:23.607 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-08-15T16:58:23.648 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T16:58:23.864 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager. 2024-08-15T16:58:23.881 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T16:58:23.947 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:23.968 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libnm0:amd64. 2024-08-15T16:58:23.988 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T16:58:24.018 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:24.421 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-08-15T16:58:24.439 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../14-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:24.479 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:24.687 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ppp. 2024-08-15T16:58:24.705 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T16:58:24.815 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libqmi-proxy. 2024-08-15T16:58:24.833 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../15-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:24.915 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:24.976 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T16:58:25.167 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-08-15T16:58:25.186 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../16-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-08-15T16:58:25.234 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:25.343 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T16:58:25.360 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T16:58:25.401 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T16:58:25.779 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-08-15T16:58:25.795 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../17-libteamdctl0_1.30-1_amd64.deb ... 2024-08-15T16:58:25.837 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-08-15T16:58:26.240 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package modemmanager. 2024-08-15T16:58:26.259 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../18-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-08-15T16:58:26.308 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:26.631 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T16:58:26.651 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T16:58:26.774 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T16:58:26.971 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package wpasupplicant. 2024-08-15T16:58:26.988 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../19-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-08-15T16:58:27.029 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T16:58:27.043 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T16:58:27.061 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T16:58:27.101 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T16:58:27.479 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T16:58:27.496 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T16:58:27.548 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T16:58:27.584 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package network-manager. 2024-08-15T16:58:27.600 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../20-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-08-15T16:58:27.675 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:27.875 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:28.185 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T16:58:28.311 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T16:58:28.437 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T16:58:29.077 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T16:58:29.077 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T16:58:29.110 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package ppp. 2024-08-15T16:58:29.128 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../21-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-08-15T16:58:29.348 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:29.401 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T16:58:29.474 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T16:58:29.608 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:29.734 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T16:58:29.739 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package pptp-linux. 2024-08-15T16:58:29.756 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../22-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-08-15T16:58:29.820 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-08-15T16:58:29.861 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:29.987 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T16:58:30.209 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package network-manager-pptp. 2024-08-15T16:58:30.227 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../23-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-08-15T16:58:30.285 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-08-15T16:58:30.562 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-08-15T16:58:30.579 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../24-usb-modeswitch-data_20191128-3_all.deb ... 2024-08-15T16:58:30.620 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-08-15T16:58:30.956 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T16:58:31.023 INFO:teuthology.orchestra.run.smithi146.stdout:Selecting previously unselected package usb-modeswitch. 2024-08-15T16:58:31.040 INFO:teuthology.orchestra.run.smithi146.stdout:Preparing to unpack .../25-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-08-15T16:58:31.092 INFO:teuthology.orchestra.run.smithi146.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T16:58:31.193 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T16:58:31.288 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T16:58:31.353 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:31.427 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:31.479 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T16:58:31.596 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T16:58:31.754 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-08-15T16:58:31.781 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T16:58:31.880 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-08-15T16:58:31.957 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T16:58:31.997 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-08-15T16:58:32.084 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:32.210 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:32.336 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T16:58:32.422 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T16:58:32.422 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-08-15T16:58:32.504 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:32.646 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:32.858 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-08-15T16:58:32.987 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-08-15T16:58:33.113 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-08-15T16:58:33.264 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout:which means they are currently configured by ifupdown: 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout:- enp3s0f1 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T16:58:33.365 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:33.390 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:33.526 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T16:58:33.537 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-08-15T16:58:33.869 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T16:58:34.391 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T16:58:34.539 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-08-15T16:58:34.780 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up dump (0.4b46-6) ... 2024-08-15T16:58:34.875 INFO:teuthology.orchestra.run.smithi146.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-08-15T16:58:34.883 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:34.940 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:35.068 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T16:58:35.082 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-08-15T16:58:35.209 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-08-15T16:58:35.210 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:35.334 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-08-15T16:58:35.336 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:35.437 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:35.538 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:35.553 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-08-15T16:58:35.679 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:35.794 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T16:58:35.794 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T16:58:35.830 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:35.956 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-08-15T16:58:36.124 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:36.275 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-08-15T16:58:36.333 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T16:58:36.451 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T16:58:36.577 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T16:58:36.761 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T16:58:36.999 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:36.999 INFO:teuthology.orchestra.run.smithi146.stdout:The following network interfaces were found in /etc/network/interfaces 2024-08-15T16:58:37.000 INFO:teuthology.orchestra.run.smithi146.stdout:which means they are currently configured by ifupdown: 2024-08-15T16:58:37.000 INFO:teuthology.orchestra.run.smithi146.stdout:- enp3s0f1 2024-08-15T16:58:37.000 INFO:teuthology.orchestra.run.smithi146.stdout:If you want to manage those interfaces with NetworkManager instead 2024-08-15T16:58:37.000 INFO:teuthology.orchestra.run.smithi146.stdout:remove their configuration from /etc/network/interfaces. 2024-08-15T16:58:37.000 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:37.155 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-08-15T16:58:37.197 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T16:58:37.464 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-08-15T16:58:37.967 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-08-15T16:58:38.420 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-08-15T16:58:38.567 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-08-15T16:58:38.706 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-08-15T16:58:38.865 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:38.991 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-08-15T16:58:39.209 INFO:teuthology.orchestra.run.smithi146.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-08-15T16:58:39.857 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T16:58:39.858 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-08-15T16:58:40.378 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-08-15T16:58:40.479 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-08-15T16:58:40.605 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-08-15T16:58:40.781 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-08-15T16:58:40.916 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T16:58:41.016 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T16:58:41.066 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-08-15T16:58:41.862 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T16:58:41.867 DEBUG:teuthology.parallel:result is None 2024-08-15T16:58:44.960 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-08-15T16:58:45.060 INFO:teuthology.orchestra.run.smithi146.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-08-15T16:58:45.905 INFO:teuthology.orchestra.run.smithi146.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-08-15T16:58:45.910 DEBUG:teuthology.parallel:result is None 2024-08-15T16:58:45.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T16:58:46.052 DEBUG:teuthology.orchestra.run.smithi146:> dpkg-query -W -f '${Version}' ceph 2024-08-15T16:58:46.076 INFO:teuthology.orchestra.run.smithi146.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T16:58:46.077 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T16:58:46.077 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T16:58:46.079 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=2c991c196a651f8ef0f36c7763ec87a241b25549 2024-08-15T16:58:46.215 DEBUG:teuthology.orchestra.run.smithi161:> dpkg-query -W -f '${Version}' ceph 2024-08-15T16:58:46.238 INFO:teuthology.orchestra.run.smithi161.stdout:17.2.7-1375-g2c991c19-1focal 2024-08-15T16:58:46.238 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375-g2c991c19-1focal 2024-08-15T16:58:46.239 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375-g2c991c19-1focal is installed. 2024-08-15T16:58:46.241 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-15T16:58:46.241 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.241 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T16:58:46.258 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:46.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-15T16:58:46.294 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-15T16:58:46.294 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.295 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T16:58:46.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T16:58:46.321 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:46.321 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-08-15T16:58:46.337 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-15T16:58:46.353 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-15T16:58:46.354 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.354 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T16:58:46.378 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T16:58:46.390 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:46.390 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-15T16:58:46.412 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-15T16:58:46.422 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-15T16:58:46.422 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.422 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T16:58:46.438 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T16:58:46.495 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:46.495 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-08-15T16:58:46.511 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-15T16:58:46.568 INFO:teuthology.run_tasks:Running task ceph... 2024-08-15T16:58:46.660 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-15T16:58:46.660 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /var/log/ceph 2024-08-15T16:58:46.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2024-08-15T16:58:46.678 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-15T16:58:46.679 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T16:58:46.721 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-15T16:58:46.737 INFO:tasks.ceph:Creating extra log directories... 2024-08-15T16:58:46.737 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T16:58:46.739 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-15T16:58:46.801 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-15T16:58:46.802 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-15T16:58:46.802 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-15T16:58:46.802 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.802 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T16:58:46.809 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T16:58:46.847 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:46.847 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-15T16:58:46.855 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-15T16:58:46.899 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-15T16:58:46.899 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-15T16:58:46.899 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7856960', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5995, 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'targets': {'smithi146.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2c991c196a651f8ef0f36c7763ec87a241b25549', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-15_16:34:53', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-15T16:58:46.900 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T16:58:46.903 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-15T16:58:46.942 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T16:58:46.953 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2024-08-15T16:58:46.968 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:46.969 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T16:58:46.973 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T16:58:46.973 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-08-15T16:58:47.022 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T16:58:47.022 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.022 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-08-15T16:58:47.022 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.022 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-15 16:54:38.912044488 +0000 2024-08-15T16:58:47.023 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-15 16:54:38.708063878 +0000 2024-08-15T16:58:47.023 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-15 16:54:38.708063878 +0000 2024-08-15T16:58:47.023 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-08-15T16:58:47.023 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T16:58:47.034 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-15T16:58:47.034 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-15T16:58:47.034 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000432077 s, 1.2 MB/s 2024-08-15T16:58:47.035 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T16:58:47.044 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-08-15T16:58:47.094 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T16:58:47.094 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.094 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-08-15T16:58:47.094 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.094 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-15 16:54:39.332004895 +0000 2024-08-15T16:58:47.095 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-15 16:54:39.140022954 +0000 2024-08-15T16:58:47.095 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-15 16:54:39.140022954 +0000 2024-08-15T16:58:47.095 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-08-15T16:58:47.095 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T16:58:47.107 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-15T16:58:47.107 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-15T16:58:47.107 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000406424 s, 1.3 MB/s 2024-08-15T16:58:47.108 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T16:58:47.116 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-15 16:54:39.779962761 +0000 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-15 16:54:39.567982701 +0000 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-15 16:54:39.567982701 +0000 2024-08-15T16:58:47.122 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-08-15T16:58:47.123 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T16:58:47.176 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-15T16:58:47.176 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-15T16:58:47.176 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000375692 s, 1.4 MB/s 2024-08-15T16:58:47.177 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T16:58:47.185 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-08-15T16:58:47.191 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T16:58:47.191 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 6h/6d Inode: 380 Links: 1 2024-08-15T16:58:47.191 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.192 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-08-15 16:54:40.195924205 +0000 2024-08-15T16:58:47.192 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-08-15 16:54:39.995942447 +0000 2024-08-15T16:58:47.192 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-08-15 16:54:39.995942447 +0000 2024-08-15T16:58:47.192 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: - 2024-08-15T16:58:47.192 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T16:58:47.206 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-08-15T16:58:47.206 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-08-15T16:58:47.206 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000368229 s, 1.4 MB/s 2024-08-15T16:58:47.206 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T16:58:47.256 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:47.256 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:47.256 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-08-15T16:58:47.263 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-15T16:58:47.263 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 360 Links: 1 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-15 16:54:37.554856703 +0000 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-15 16:54:37.342861592 +0000 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-15 16:54:37.342861592 +0000 2024-08-15T16:58:47.311 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-08-15T16:58:47.311 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-15T16:58:47.326 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-15T16:58:47.326 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-15T16:58:47.326 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000426359 s, 1.2 MB/s 2024-08-15T16:58:47.327 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-15T16:58:47.335 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-15 16:54:37.998846464 +0000 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-15 16:54:37.770851723 +0000 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-15 16:54:37.770851723 +0000 2024-08-15T16:58:47.383 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-08-15T16:58:47.384 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-15T16:58:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-15T16:58:47.395 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-15T16:58:47.396 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000406753 s, 1.3 MB/s 2024-08-15T16:58:47.397 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-15T16:58:47.405 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-08-15T16:58:47.410 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-15 16:54:38.434836408 +0000 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-15 16:54:38.222841299 +0000 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-15 16:54:38.222841299 +0000 2024-08-15T16:58:47.411 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-08-15T16:58:47.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-15T16:58:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-15T16:58:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-15T16:58:47.423 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000412424 s, 1.2 MB/s 2024-08-15T16:58:47.424 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-15T16:58:47.476 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-08-15T16:58:47.484 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-15T16:58:47.484 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-15T16:58:47.484 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-08-15T16:58:47.485 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-15T16:58:47.485 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-08-15 16:54:38.894825799 +0000 2024-08-15T16:58:47.485 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-08-15 16:54:38.690830505 +0000 2024-08-15T16:58:47.485 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-08-15 16:54:38.690830505 +0000 2024-08-15T16:58:47.485 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2024-08-15T16:58:47.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-15T16:58:47.497 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-08-15T16:58:47.498 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-08-15T16:58:47.498 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000311377 s, 1.6 MB/s 2024-08-15T16:58:47.499 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-15T16:58:47.548 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:47.548 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T16:58:47.548 INFO:tasks.ceph:Generating config... 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] debug client = 20 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] debug ms = 1 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] pid file = 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-15T16:58:47.550 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-15T16:58:47.550 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-15T16:58:47.550 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-15T16:58:47.550 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-15T16:58:47.551 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-15T16:58:47.552 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-15T16:58:47.552 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-15T16:58:47.552 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-15T16:58:47.553 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-15T16:58:47.553 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-15T16:58:47.553 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-15T16:58:47.553 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-15T16:58:47.553 INFO:tasks.ceph:Setting up mon.a... 2024-08-15T16:58:47.553 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-15T16:58:47.588 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.keyring 2024-08-15T16:58:47.591 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-15T16:58:47.624 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T16:58:47.684 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.146'), ('mon.b', '172.21.15.161'), ('mon.c', '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]')] 2024-08-15T16:58:47.684 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.146,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-15T16:58:47.686 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:47.686 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T16:58:47.692 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.146 --add b 172.21.15.161 --addv c '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: generated fsid 10958773-fd73-437e-aa56-f078e9b084bc 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:setting min_mon_release = octopus 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:epoch 0 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:fsid 10958773-fd73-437e-aa56-f078e9b084bc 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:last_changed 2024-08-15T16:58:47.765884+0000 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:created 2024-08-15T16:58:47.765884+0000 2024-08-15T16:58:47.775 INFO:teuthology.orchestra.run.smithi146.stdout:min_mon_release 15 (octopus) 2024-08-15T16:58:47.776 INFO:teuthology.orchestra.run.smithi146.stdout:election_strategy: 1 2024-08-15T16:58:47.776 INFO:teuthology.orchestra.run.smithi146.stdout:0: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.a 2024-08-15T16:58:47.776 INFO:teuthology.orchestra.run.smithi146.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-08-15T16:58:47.776 INFO:teuthology.orchestra.run.smithi146.stdout:2: [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] mon.c 2024-08-15T16:58:47.776 INFO:teuthology.orchestra.run.smithi146.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-15T16:58:47.777 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-15T16:58:47.784 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 10958773-fd73-437e-aa56-f078e9b084bc... 2024-08-15T16:58:47.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T16:58:47.788 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-15T16:58:47.816 INFO:teuthology.orchestra.run.smithi146.stdout:[global] 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: chdir = "" 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: auth supported = cephx 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: filestore xattr use omap = true 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: mon clock drift allowed = 1.000 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: osd crush chooseleaf type = 0 2024-08-15T16:58:47.817 INFO:teuthology.orchestra.run.smithi146.stdout: auth debug = true 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on old message = true 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: ms die on bug = true 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: mon pg warn max object skew = 0 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.818 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default size = 2 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow primary affinity = true 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd allow pg remap = true 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on legacy crush tunables = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on crush straw calc version zero = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on no sortbitwise = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on osd down out interval zero = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon warn on too few osds = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: mon_allow_pool_size_one = true 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.819 INFO:teuthology.orchestra.run.smithi146.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: osd default data pool replay window = 5 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: mon allow pool delete = true 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: mon cluster log file level = debug 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: debug asserts on shutdown = true 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: mon health detail to clog = false 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: mon host = "172.21.15.146,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2024-08-15T16:58:47.820 INFO:teuthology.orchestra.run.smithi146.stdout: fsid = 10958773-fd73-437e-aa56-f078e9b084bc 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout:[osd] 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd journal size = 100 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub load threshold = 5.0 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd scrub max interval = 600 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd mclock profile = high_recovery_ops 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd recover clone overlap = true 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd recovery max chunk = 1048576 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug shutdown = true 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug op order = true 2024-08-15T16:58:47.821 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug verify stray on activate = true 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: osd open classes on start = true 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug pg log writeout = true 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: osd deep scrub update digest min age = 30 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: osd map max advance = 10 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: journal zero on create = true 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: filestore ondisk finisher threads = 3 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: filestore apply finisher threads = 3 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.822 INFO:teuthology.orchestra.run.smithi146.stdout: bdev debug aio = true 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: osd debug misdirected ops = true 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: bdev async discard = True 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: bdev enable discard = True 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore allocator = bitmap 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore block size = 96636764160 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: bluestore fsck on mount = True 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluefs = 1/20 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: debug bluestore = 1/20 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: debug osd = 20 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: debug rocksdb = 4/10 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd full ratio = 0.9 2024-08-15T16:58:47.823 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd nearfull ratio = 0.8 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: osd failsafe full ratio = 0.95 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: osd objectstore = bluestore 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr] 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: debug mgr = 20 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: mgr/telemetry/nag = false 2024-08-15T16:58:47.824 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: debug finisher = 20 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout:[mon] 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: debug mon = 20 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: debug paxos = 20 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: debug auth = 20 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: mon data avail warn = 5 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: mon mgr mkfs grace = 240 2024-08-15T16:58:47.825 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min pgs per osd = 4 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd reporter subtree level = osd 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: mon osd prime pg temp = true 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: mon reweight min bytes per osd = 10 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T16:58:47.826 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: # 1m isn't quite enough 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: mon_down_mkfs_grace = 2m 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: mon_warn_on_filestore_osds = false 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: mon op complaint time = 120 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout:[client] 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: rgw cache enabled = true 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable ops log = true 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: rgw enable usage log = true 2024-08-15T16:58:47.827 INFO:teuthology.orchestra.run.smithi146.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: client mount timeout = 600 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: debug client = 20 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: pid file = "" 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.a] 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.b] 2024-08-15T16:58:47.828 INFO:teuthology.orchestra.run.smithi146.stdout:[mon.c] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout:[mds] 2024-08-15T16:58:47.829 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds = 20 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: debug mds balancer = 20 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: debug ms = 1 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug frag = True 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: mds debug scatterstat = True 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: mds op complaint time = 180 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: mds verify scatter = True 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: osd op complaint time = 180 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: rados mon op timeout = 900 2024-08-15T16:58:47.830 INFO:teuthology.orchestra.run.smithi146.stdout: rados osd op timeout = 900 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.832 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2024-08-15T16:58:47.833 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2024-08-15T16:58:47.834 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.146,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 10958773-fd73-437e-aa56-f078e9b084bc 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2024-08-15T16:58:47.835 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2024-08-15T16:58:47.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bdev async discard = True 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bdev enable discard = True 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore allocator = bitmap 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2024-08-15T16:58:47.837 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 1/20 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 1/20 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 4/10 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 0.95 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = bluestore 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2024-08-15T16:58:47.838 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug finisher = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-08-15T16:58:47.839 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-15T16:58:47.840 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: mon op complaint time = 120 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2024-08-15T16:58:47.841 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: client mount timeout = 600 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = "" 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-08-15T16:58:47.842 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout:[mds] 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds = 20 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds balancer = 20 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug frag = True 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug scatterstat = True 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: mds op complaint time = 180 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: mds verify scatter = True 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-08-15T16:58:47.843 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-08-15T16:58:47.844 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-15T16:58:47.844 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-15T16:58:47.923 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-15T16:58:47.923 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:47.923 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-15T16:58:47.931 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:47.931 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-15T16:58:47.937 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:58:47.938 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:47.938 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T16:58:47.938 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T16:58:48.006 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:58:48.006 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:48.012 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:58:48.012 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:48.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-15T16:58:48.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-15T16:58:48.039 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:58:48.040 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:48.090 INFO:tasks.ceph:Setting up mon nodes... 2024-08-15T16:58:48.090 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-15T16:58:48.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T16:58:48.135 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-15T16:58:48.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T16:58:48.181 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-15T16:58:48.183 INFO:tasks.ceph:Setting up mds nodes... 2024-08-15T16:58:48.184 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T16:58:48.227 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-15T16:58:48.230 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-15T16:58:48.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T16:58:48.285 INFO:teuthology.orchestra.run.smithi146.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-15T16:58:48.287 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-15T16:58:48.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T16:58:48.392 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-15T16:58:48.395 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-15T16:58:48.451 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T16:58:48.496 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-15T16:58:48.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-15T16:58:48.511 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-15T16:58:48.511 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-15T16:58:48.548 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-15T16:58:48.562 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-08-15T16:58:48.592 INFO:teuthology.orchestra.run.smithi146.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-08-15T16:58:48.603 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-15T16:58:48.638 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-15T16:58:48.652 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-08-15T16:58:48.732 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-08-15T16:58:48.744 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-15T16:58:48.744 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi146.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-15T16:58:48.745 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-15T16:58:48.761 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:48.762 INFO:tasks.ceph:role: osd.0 2024-08-15T16:58:48.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:58:48.762 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:58:49.341 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:58:49.342 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:58:49.342 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:58:49.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-08-15T16:58:49.343 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-15T16:58:49.431 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-15T16:58:49.443 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:58:49.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:58:49.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-15T16:58:49.505 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:49.505 INFO:tasks.ceph:role: osd.1 2024-08-15T16:58:49.505 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:58:49.505 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:58:50.022 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:58:50.023 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:58:50.023 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:58:50.023 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:58:50.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-08-15T16:58:50.024 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-15T16:58:50.065 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-15T16:58:50.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:58:50.121 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:58:50.121 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-15T16:58:50.132 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:50.132 INFO:tasks.ceph:role: osd.2 2024-08-15T16:58:50.132 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:58:50.132 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:58:50.614 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:58:50.615 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:58:50.615 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:58:50.615 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:58:50.616 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-08-15T16:58:50.616 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-15T16:58:50.670 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-15T16:58:50.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:58:50.682 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:58:50.682 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-15T16:58:50.692 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-15T16:58:50.692 INFO:tasks.ceph:role: osd.3 2024-08-15T16:58:50.692 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T16:58:50.692 DEBUG:teuthology.orchestra.run.smithi146:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T16:58:51.192 INFO:teuthology.orchestra.run.smithi146.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:58:51.192 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout: = reflink=1 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:58:51.193 INFO:teuthology.orchestra.run.smithi146.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:58:51.195 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi146.front.sepia.ceph.com -o noatime 2024-08-15T16:58:51.195 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-15T16:58:51.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-15T16:58:51.254 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:58:51.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:58:51.256 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:51.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.318+0000 7f1b4d2b03c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-15T16:58:51.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.318+0000 7f1b4d2b03c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-15T16:58:51.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.318+0000 7f1b4d2b03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T16:58:51.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.318+0000 7f1b4d2b03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T16:58:51.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.318+0000 7f1b4d2b03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-15T16:58:51.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.322+0000 7f1b4d2b03c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-15T16:58:51.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:51.882+0000 7f1b4d2b03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:52.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:52.678+0000 7f1b4d2b03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:53.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:53.462+0000 7f1b4d2b03c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:54.336 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-15T16:58:54.352 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:54.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-15T16:58:54.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-15T16:58:54.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T16:58:54.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T16:58:54.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-15T16:58:54.437 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.426+0000 7f33da4583c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-15T16:58:54.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:54.985+0000 7f33da4583c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:55.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:55.769+0000 7f33da4583c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:56.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:56.557+0000 7f33da4583c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:57.431 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-15T16:58:57.447 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:58:57.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.469+0000 7f0c84c293c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-15T16:58:57.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.469+0000 7f0c84c293c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-15T16:58:57.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.469+0000 7f0c84c293c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T16:58:57.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.469+0000 7f0c84c293c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T16:58:57.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.469+0000 7f0c84c293c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-15T16:58:57.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:57.473+0000 7f0c84c293c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-15T16:58:58.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:58.037+0000 7f0c84c293c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:58.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:58.813+0000 7f0c84c293c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:58:59.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:58:59.605+0000 7f0c84c293c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:00.475 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-15T16:59:00.536 DEBUG:teuthology.orchestra.run.smithi146:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:00.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-15T16:59:00.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-15T16:59:00.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T16:59:00.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T16:59:00.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-15T16:59:00.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:00.565+0000 7f22016b83c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-15T16:59:01.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:01.125+0000 7f22016b83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:01.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:01.905+0000 7f22016b83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:02.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:02.685+0000 7f22016b83c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:03.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-15T16:59:03.559 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-15T16:59:03.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T16:59:03.577 INFO:tasks.ceph:role: osd.4 2024-08-15T16:59:03.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:59:03.577 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:59:04.128 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:59:04.130 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-08-15T16:59:04.130 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-15T16:59:04.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-15T16:59:04.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:04.222 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:59:04.222 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-15T16:59:04.233 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T16:59:04.233 INFO:tasks.ceph:role: osd.5 2024-08-15T16:59:04.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:59:04.233 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-15T16:59:04.754 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:59:04.754 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:59:04.754 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:59:04.754 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:59:04.755 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:59:04.756 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-08-15T16:59:04.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-15T16:59:04.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-15T16:59:04.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:04.846 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:59:04.846 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-15T16:59:04.862 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T16:59:04.862 INFO:tasks.ceph:role: osd.6 2024-08-15T16:59:04.862 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:59:04.862 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-15T16:59:05.315 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:59:05.315 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:59:05.315 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:59:05.315 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:59:05.316 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:59:05.318 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-08-15T16:59:05.318 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-15T16:59:05.379 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-15T16:59:05.390 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:59:05.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:05.391 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-15T16:59:05.400 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-15T16:59:05.400 INFO:tasks.ceph:role: osd.7 2024-08-15T16:59:05.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T16:59:05.401 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-15T16:59:05.907 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-08-15T16:59:05.908 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-15T16:59:05.908 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-15T16:59:05.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-08-15T16:59:05.909 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-15T16:59:05.967 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-15T16:59:05.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:05.978 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2024-08-15T16:59:05.979 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:06.053 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.040+0000 7facca3473c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-15T16:59:06.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.040+0000 7facca3473c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-15T16:59:06.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.040+0000 7facca3473c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T16:59:06.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.040+0000 7facca3473c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T16:59:06.054 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.040+0000 7facca3473c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-15T16:59:06.060 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.048+0000 7facca3473c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-15T16:59:06.624 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:06.608+0000 7facca3473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:07.402 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:07.388+0000 7facca3473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:08.183 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:08.168+0000 7facca3473c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:09.028 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-15T16:59:09.044 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:09.124 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.108+0000 7f43409533c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-15T16:59:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.112+0000 7f43409533c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-15T16:59:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.112+0000 7f43409533c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T16:59:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.112+0000 7f43409533c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T16:59:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.112+0000 7f43409533c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-15T16:59:09.125 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.112+0000 7f43409533c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-15T16:59:09.684 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:09.668+0000 7f43409533c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:10.468 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:10.452+0000 7f43409533c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:11.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:11.248+0000 7f43409533c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:12.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-15T16:59:12.126 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:12.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.152+0000 7fae44a113c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-15T16:59:12.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.156+0000 7fae44a113c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-15T16:59:12.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.156+0000 7fae44a113c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T16:59:12.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.156+0000 7fae44a113c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T16:59:12.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.156+0000 7fae44a113c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-15T16:59:12.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.156+0000 7fae44a113c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-15T16:59:12.724 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:12.708+0000 7fae44a113c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:13.509 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:13.496+0000 7fae44a113c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:14.296 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:14.280+0000 7fae44a113c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:15.148 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-15T16:59:15.164 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:15.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-15T16:59:15.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-15T16:59:15.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T16:59:15.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T16:59:15.249 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-15T16:59:15.250 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.236+0000 7fc2839a43c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-15T16:59:15.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:15.788+0000 7fc2839a43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:16.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:16.580+0000 7fc2839a43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:16.901 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T16:59:16.905 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T16:59:17.376 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:17.360+0000 7fc2839a43c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-15T16:59:18.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-15T16:59:18.244 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-15T16:59:18.244 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-15T16:59:18.261 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.261 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-15T16:59:18.276 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.276 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-15T16:59:18.291 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.291 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-15T16:59:18.305 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-15T16:59:18.362 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.363 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-15T16:59:18.375 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-15T16:59:18.386 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-15T16:59:18.402 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.402 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-15T16:59:18.417 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.418 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-15T16:59:18.431 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.431 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-15T16:59:18.444 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.445 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-15T16:59:18.503 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-15T16:59:18.514 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.518 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-15T16:59:18.528 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.528 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-15T16:59:18.534 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:18.534 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-08-15T16:59:18.582 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.582 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-15T16:59:18.588 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:18.588 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-08-15T16:59:18.595 INFO:tasks.ceph:Adding keys to all mons... 2024-08-15T16:59:18.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T16:59:18.597 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.y] 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75msKqIBxAA4N3PyxiT1fLJrG8xeSHh0g== 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.a] 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75maQcKDRAAWdaN3na/L2qB437SOQKc5w== 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.c] 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mMlx3EBAA7H9f6HWTvB44NXUPLs9bjQ== 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.0] 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDLM75mfEcDExAAAwHP4sfPsT0ZgUZ7AHH5PA== 2024-08-15T16:59:18.639 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.1] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDOM75mOvtwGRAAlfIQF/O9oyllcVWZTeOlwg== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.2] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDRM75mOyI4HBAAp8V4uXtcNax/1vNYkp9XTA== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.3] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDUM75m9kjbIRAAhlj6FuHE+AvOaNzV/38Q+g== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[mgr.x] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mCt0hChAApiDTDwbfZMW6wR3FSTc0cg== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.b] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mHUC7FhAAq9gwtEFOiyqrnPvAeu9Oqw== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[mds.d] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mcN3qHBAA5fqtrfo5CP4R9JrI4toBPg== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.4] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDaM75m4xx/AhAA2ZCxJiS34e6Mpappwdir4A== 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.5] 2024-08-15T16:59:18.640 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDdM75mZ5a6BhAAs6AtJ5GJlE49U9sobLhKgQ== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.6] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDgM75mCf1eCRAAGDg04874V5e2DzHUNM0Q5A== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[osd.7] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDjM75mC8QmDhAAPpGC18CrTOzLgairACuZOg== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[client.0] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mEyclIBAAjIGu9uX2cilmZc1BD1bFbA== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[client.2] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mwQPFIhAAy72vH4sSpT+chLV9GuGthg== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75m7PlkJRAAnPA93Ew9D+wE4HbBHV98Zg== 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout:[client.3] 2024-08-15T16:59:18.641 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQDIM75mSRn9KhAAcY7ez+KrfzxZ/w00NzSg+Q== 2024-08-15T16:59:18.650 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2024-08-15T16:59:18.650 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75msKqIBxAA4N3PyxiT1fLJrG8xeSHh0g== 2024-08-15T16:59:18.650 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75maQcKDRAAWdaN3na/L2qB437SOQKc5w== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mMlx3EBAA7H9f6HWTvB44NXUPLs9bjQ== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDLM75mfEcDExAAAwHP4sfPsT0ZgUZ7AHH5PA== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDOM75mOvtwGRAAlfIQF/O9oyllcVWZTeOlwg== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDRM75mOyI4HBAAp8V4uXtcNax/1vNYkp9XTA== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDUM75m9kjbIRAAhlj6FuHE+AvOaNzV/38Q+g== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mCt0hChAApiDTDwbfZMW6wR3FSTc0cg== 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-08-15T16:59:18.651 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mHUC7FhAAq9gwtEFOiyqrnPvAeu9Oqw== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mcN3qHBAA5fqtrfo5CP4R9JrI4toBPg== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDaM75m4xx/AhAA2ZCxJiS34e6Mpappwdir4A== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDdM75mZ5a6BhAAs6AtJ5GJlE49U9sobLhKgQ== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDgM75mCf1eCRAAGDg04874V5e2DzHUNM0Q5A== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDjM75mC8QmDhAAPpGC18CrTOzLgairACuZOg== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mEyclIBAAjIGu9uX2cilmZc1BD1bFbA== 2024-08-15T16:59:18.652 INFO:teuthology.orchestra.run.smithi161.stdout:[client.2] 2024-08-15T16:59:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mwQPFIhAAy72vH4sSpT+chLV9GuGthg== 2024-08-15T16:59:18.653 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-08-15T16:59:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75m7PlkJRAAnPA93Ew9D+wE4HbBHV98Zg== 2024-08-15T16:59:18.653 INFO:teuthology.orchestra.run.smithi161.stdout:[client.3] 2024-08-15T16:59:18.653 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQDIM75mSRn9KhAAcY7ez+KrfzxZ/w00NzSg+Q== 2024-08-15T16:59:18.654 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T16:59:18.689 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T16:59:18.735 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:18.738 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:18.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:18.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:18.877 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:18.879 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:18.927 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:18.929 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:18.973 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:18.975 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.018 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.107 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T16:59:19.110 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-15T16:59:19.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:19.158 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:19.204 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:19.206 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-15T16:59:19.292 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.294 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.340 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.342 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.385 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.472 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.474 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-15T16:59:19.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.564 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.652 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.702 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-15T16:59:19.750 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-15T16:59:19.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-15T16:59:19.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T16:59:19.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-15T16:59:19.874 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-15T16:59:19.891 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T16:59:19.942 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-15T16:59:19.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-15T16:59:20.021 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-15T16:59:20.074 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-15T16:59:20.087 DEBUG:teuthology.orchestra.run.smithi146:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:20.089 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-15T16:59:20.094 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-15T16:59:20.095 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-15T16:59:20.095 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-15T16:59:20.137 INFO:tasks.ceph.mon.a:Started 2024-08-15T16:59:20.137 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-15T16:59:20.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-15T16:59:20.140 INFO:tasks.ceph.mon.b:Started 2024-08-15T16:59:20.140 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-15T16:59:20.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-15T16:59:20.142 INFO:tasks.ceph.mon.c:Started 2024-08-15T16:59:20.142 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-15T16:59:20.143 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-15T16:59:20.143 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-15T16:59:20.145 INFO:tasks.ceph.mgr.y:Started 2024-08-15T16:59:20.145 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-15T16:59:20.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-15T16:59:20.147 INFO:tasks.ceph.mgr.x:Started 2024-08-15T16:59:20.148 INFO:tasks.ceph:Setting crush tunables to default 2024-08-15T16:59:20.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph osd crush tunables default 2024-08-15T16:59:20.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.249+0000 7ff2dd473700 1 Processor -- start 2024-08-15T16:59:20.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.249+0000 7ff2dd473700 1 -- start start 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2dd473700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2dd473700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2dd473700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80b01a0 con 0x7ff2d8155d60 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2dd473700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d815ab00 con 0x7ff2d806d7e0 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2dd473700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d806dc80 con 0x7ff2d8156130 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d6ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff2d8155d60 0x7ff2d80afa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34560/0 (socket says 172.21.15.146:34560) 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 learned_addr learned my addr 172.21.15.146:0/1058202404 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T16:59:20.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T16:59:20.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.253+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T16:59:20.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T16:59:20.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.457+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T16:59:20.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:20.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:20.861+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d67fc700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:21.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:21.665+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 msgr2=0x7ff2d806d2a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.161:3300/0 conn(0x7ff2d8156130 0x7ff2d806d2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 -- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 msgr2=0x7ff2d80a8e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 --2- 172.21.15.146:0/1058202404 >> v2:172.21.15.146:3300/0 conn(0x7ff2d806d7e0 0x7ff2d80a8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 -- 172.21.15.146:0/1058202404 >> v1:172.21.15.146:6789/0 conn(0x7ff2d8155d60 legacy=0x7ff2d80afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 -- 172.21.15.146:0/1058202404 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d815a820 con 0x7ff2c4006190 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d4ff9700 1 -- 172.21.15.146:0/1058202404 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815ab00 con 0x7ff2c40036c0 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T16:59:23.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:23.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:23.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.253+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-15T16:59:23.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.457+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:23.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.457+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T16:59:23.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.457+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:23.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.457+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:23.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.457+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-15T16:59:23.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.861+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:23.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.861+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T16:59:23.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.861+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:23.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.861+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:23.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:23.861+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-15T16:59:24.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:24.665+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:24.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:24.665+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-08-15T16:59:24.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:24.665+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-15T16:59:24.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:24.665+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-15T16:59:24.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:24.665+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-15T16:59:25.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384680113 0 0) 0x7ff2d815a820 con 0x7ff2c4006190 2024-08-15T16:59:25.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c0003570 con 0x7ff2c4006190 2024-08-15T16:59:25.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482380519 0 0) 0x7ff2c0003570 con 0x7ff2c4006190 2024-08-15T16:59:25.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 msgr2=0x7ff2c4005b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T16:59:25.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c40036c0 0x7ff2c4005b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c400af80 con 0x7ff2c4006190 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7ff2c80097a0 con 0x7ff2c4006190 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 >> v1:172.21.15.161:6789/0 conn(0x7ff2c4006190 legacy=0x7ff2c4008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d77fe700 1 -- 172.21.15.146:0/1058202404 reap_dead start 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 0x7ff2c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0004810 0x7ff2c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 0x7ff2c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d815ab00 con 0x7ff2c0004810 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000c250 con 0x7ff2c00052e0 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c000c410 con 0x7ff2c0003a60 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 0x7ff2c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 0x7ff2c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0004810 0x7ff2c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 msgr2=0x7ff2c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 0x7ff2c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0004810 msgr2=0x7ff2c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0004810 0x7ff2c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/1058202404 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0003570 con 0x7ff2c0003a60 2024-08-15T16:59:25.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 0x7ff2c000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:25.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.345+0000 7ff2d6ffd700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 0x7ff2c0003e50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0095f0 tx=0x7ff2cc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d74d7d1ae37ed1b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc01e030 con 0x7ff2c0003a60 2024-08-15T16:59:25.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc001670 con 0x7ff2c0003a60 2024-08-15T16:59:25.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2d5ffb700 1 -- 172.21.15.146:0/1058202404 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc009a30 con 0x7ff2c0003a60 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 -- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 msgr2=0x7ff2c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 0x7ff2c0003e50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0095f0 tx=0x7ff2cc00b040 comp rx=0 tx=0).stop 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 -- 172.21.15.146:0/1058202404 shutdown_connections 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c00052e0 0x7ff2c000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0004810 0x7ff2c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 --2- 172.21.15.146:0/1058202404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c0003a60 0x7ff2c0003e50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 -- 172.21.15.146:0/1058202404 >> 172.21.15.146:0/1058202404 conn(0x7ff2d8066e00 msgr2=0x7ff2d8067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:25.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 -- 172.21.15.146:0/1058202404 shutdown_connections 2024-08-15T16:59:25.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.349+0000 7ff2dd473700 1 -- 172.21.15.146:0/1058202404 wait complete. 2024-08-15T16:59:25.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 Processor -- start 2024-08-15T16:59:25.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 -- start start 2024-08-15T16:59:25.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 0x7ff2d80992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c00052e0 0x7ff2d80c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d8099810 con 0x7ff2c0003a60 2024-08-15T16:59:25.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d816a2e0 con 0x7ff2c0004810 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.353+0000 7ff2dd473700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d816ae60 con 0x7ff2c00052e0 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 0x7ff2d80992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d6ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 0x7ff2d80992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43780/0 (socket says 172.21.15.146:43780) 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:47644/0 (socket says 172.21.15.146:47644) 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d6ffd700 1 -- 172.21.15.146:0/159609231 learned_addr learned my addr 172.21.15.146:0/159609231 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:25.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d77fe700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c00052e0 0x7ff2d80c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 -- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c00052e0 msgr2=0x7ff2d80c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c00052e0 0x7ff2d80c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 -- 172.21.15.146:0/159609231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 msgr2=0x7ff2d80992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 0x7ff2d80992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 -- 172.21.15.146:0/159609231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d816a530 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2d67fc700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8015600 tx=0x7ff2c800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8018660 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c800f040 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d816a810 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8018a30 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d816a070 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff2c8003040 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff2c8023c40 con 0x7ff2c0004810 2024-08-15T16:59:25.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.357+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d816ac90 con 0x7ff2c0004810 2024-08-15T16:59:25.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.361+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff2c8028060 con 0x7ff2c0004810 2024-08-15T16:59:25.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.433+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff2d80645c0 con 0x7ff2c0004810 2024-08-15T16:59:25.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2beffd700 1 -- 172.21.15.146:0/159609231 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff2c8031090 con 0x7ff2c0004810 2024-08-15T16:59:25.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 msgr2=0x7ff2d80c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8015600 tx=0x7ff2c800b040 comp rx=0 tx=0).stop 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 shutdown_connections 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2c00052e0 0x7ff2d80c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2c0004810 0x7ff2d80c58a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 --2- 172.21.15.146:0/159609231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2c0003a60 0x7ff2d80992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 >> 172.21.15.146:0/159609231 conn(0x7ff2d8066e00 msgr2=0x7ff2d80a9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 shutdown_connections 2024-08-15T16:59:25.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:25.441+0000 7ff2dd473700 1 -- 172.21.15.146:0/159609231 wait complete. 2024-08-15T16:59:25.456 INFO:teuthology.orchestra.run.smithi146.stderr:adjusted tunables profile to default 2024-08-15T16:59:25.466 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-15T16:59:25.466 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:25.467 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-15T16:59:25.482 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:25.482 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-15T16:59:25.497 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:25.497 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-15T16:59:25.513 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-08-15T16:59:25.513 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-15T16:59:25.525 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:25.525 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-15T16:59:25.542 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:25.542 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-15T16:59:25.550 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:25.535+0000 7fe5a8472000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T16:59:25.561 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:25.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-15T16:59:25.619 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-08-15T16:59:25.619 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-15T16:59:25.630 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c 0 2024-08-15T16:59:25.737 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 Processor -- start 2024-08-15T16:59:25.738 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 -- start start 2024-08-15T16:59:25.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57c4068db0 0x7f57c4159380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 0x7f57c40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c406a3b0 con 0x7f57c4158fb0 2024-08-15T16:59:25.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c4069ee0 con 0x7f57c4068db0 2024-08-15T16:59:25.740 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57cc60c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40c9010 con 0x7f57c40c8b70 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57c4068db0 0x7f57c4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57ca3a8700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f57c4158fb0 0x7f57c40686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58684/0 (socket says 172.21.15.161:58684) 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57ca3a8700 1 -- 172.21.15.161:0/4058413899 learned_addr learned my addr 172.21.15.161:0/4058413899 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57caba9700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 0x7f57c40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c93a6700 1 -- 172.21.15.161:0/4058413899 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700477945 0 0) 0x7f57c406a3b0 con 0x7f57c4158fb0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c93a6700 1 -- 172.21.15.161:0/4058413899 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57b0003570 con 0x7f57c4158fb0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 -- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 msgr2=0x7f57c40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 0x7f57c40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 -- 172.21.15.161:0/4058413899 >> v1:172.21.15.161:6789/0 conn(0x7f57c4158fb0 legacy=0x7f57c40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 -- 172.21.15.161:0/4058413899 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40a9fe0 con 0x7f57c4068db0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57caba9700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 0x7f57c40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c9ba7700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4068db0 0x7f57c4159380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57b40098e0 tx=0x7f57b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555bd8bbac763880 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.741 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c93a6700 1 -- 172.21.15.161:0/4058413899 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4011630 con 0x7f57c4068db0 2024-08-15T16:59:25.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c93a6700 1 -- 172.21.15.161:0/4058413899 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4011790 con 0x7f57c4068db0 2024-08-15T16:59:25.742 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.723+0000 7f57c93a6700 1 -- 172.21.15.161:0/4058413899 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4011b30 con 0x7f57c4068db0 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4068db0 msgr2=0x7f57c4159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4068db0 0x7f57c4159380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57b40098e0 tx=0x7f57b40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- 172.21.15.161:0/4058413899 shutdown_connections 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c40c8b70 0x7f57c40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 --2- 172.21.15.161:0/4058413899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c4068db0 0x7f57c4159380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- 172.21.15.161:0/4058413899 >> 172.21.15.161:0/4058413899 conn(0x7f57c40c9bf0 msgr2=0x7f57c406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- 172.21.15.161:0/4058413899 shutdown_connections 2024-08-15T16:59:25.743 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- 172.21.15.161:0/4058413899 wait complete. 2024-08-15T16:59:25.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 Processor -- start 2024-08-15T16:59:25.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.727+0000 7f57cc60c700 1 -- start start 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 0x7f57c40ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c416a270 con 0x7f57c4068db0 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40cb700 con 0x7f57c40c8b70 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c40cb9e0 con 0x7f57c4158fb0 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57c9ba7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:47984/0 (socket says 172.21.15.161:47984) 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57c9ba7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:49980/0 (socket says 172.21.15.161:49980) 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 -- 172.21.15.161:0/951415775 learned_addr learned my addr 172.21.15.161:0/951415775 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ca3a8700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 0x7f57c40ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 -- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 msgr2=0x7f57c415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 -- 172.21.15.161:0/951415775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 msgr2=0x7f57c40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 0x7f57c40ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 -- 172.21.15.161:0/951415775 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009a20 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ca3a8700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 0x7f57c40ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57c9ba7700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57caba9700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00cc80 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00f040 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b4009420 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc014480 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c40cbfa0 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f57bc003020 con 0x7f57c4158fb0 2024-08-15T16:59:25.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f57bc01db00 con 0x7f57c4158fb0 2024-08-15T16:59:25.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.731+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c40645c0 con 0x7f57c4158fb0 2024-08-15T16:59:25.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.735+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f57bc018030 con 0x7f57c4158fb0 2024-08-15T16:59:25.827 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.811+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c", "id": 0} v 0) v1 -- 0x7f57c40b0580 con 0x7f57c4158fb0 2024-08-15T16:59:25.834 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57ba7fc700 1 -- 172.21.15.161:0/951415775 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f57bc02b0b0 con 0x7f57c4158fb0 2024-08-15T16:59:25.834 INFO:teuthology.orchestra.run.smithi161.stdout:0 2024-08-15T16:59:25.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 msgr2=0x7f57c415cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.835 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009fc0 tx=0x7f57bc00b040 comp rx=0 tx=0).stop 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 shutdown_connections 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c4158fb0 0x7f57c415cf00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c40c8b70 0x7f57c415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 --2- 172.21.15.161:0/951415775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57c4068db0 0x7f57c40ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 >> 172.21.15.161:0/951415775 conn(0x7f57c40c9bf0 msgr2=0x7f57c41576e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 shutdown_connections 2024-08-15T16:59:25.836 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.819+0000 7f57cc60c700 1 -- 172.21.15.161:0/951415775 wait complete. 2024-08-15T16:59:25.847 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new c36dbee1-7d4f-4553-a18a-c1a70cfc250a 1 2024-08-15T16:59:25.952 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.935+0000 7f3e4895c700 1 Processor -- start 2024-08-15T16:59:25.953 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.935+0000 7f3e4895c700 1 -- start start 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e44067030 0x7f3e44067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e440cf670 con 0x7f3e44158cc0 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e440cea60 con 0x7f3e440cfb00 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e440ced40 con 0x7f3e44067030 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e437fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e44067030 0x7f3e44067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e42ffd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3e440cfb00 0x7f3e440cb080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58688/0 (socket says 172.21.15.161:58688) 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e42ffd700 1 -- 172.21.15.161:0/437310698 learned_addr learned my addr 172.21.15.161:0/437310698 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:25.954 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e437fe700 1 -- 172.21.15.161:0/437310698 >> v1:172.21.15.161:6789/0 conn(0x7f3e440cfb00 legacy=0x7f3e440cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e437fe700 1 -- 172.21.15.161:0/437310698 >> v1:172.21.15.146:6789/0 conn(0x7f3e44158cc0 legacy=0x7f3e441590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.955 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e437fe700 1 -- 172.21.15.161:0/437310698 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e440cf020 con 0x7f3e44067030 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e437fe700 1 --2- 172.21.15.161:0/437310698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44067030 0x7f3e44067400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e340098e0 tx=0x7f3e340091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04c64158d4eb0be server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e427fc700 1 -- 172.21.15.161:0/437310698 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e34009e80 con 0x7f3e44067030 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e427fc700 1 -- 172.21.15.161:0/437310698 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3400b860 con 0x7f3e44067030 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e427fc700 1 -- 172.21.15.161:0/437310698 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400bbe0 con 0x7f3e44067030 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- 172.21.15.161:0/437310698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44067030 msgr2=0x7f3e44067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 --2- 172.21.15.161:0/437310698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44067030 0x7f3e44067400 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e340098e0 tx=0x7f3e340091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- 172.21.15.161:0/437310698 shutdown_connections 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 --2- 172.21.15.161:0/437310698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44067030 0x7f3e44067400 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- 172.21.15.161:0/437310698 >> 172.21.15.161:0/437310698 conn(0x7f3e440d0b50 msgr2=0x7f3e440ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- 172.21.15.161:0/437310698 shutdown_connections 2024-08-15T16:59:25.956 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.939+0000 7f3e4895c700 1 -- 172.21.15.161:0/437310698 wait complete. 2024-08-15T16:59:25.957 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 Processor -- start 2024-08-15T16:59:25.958 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- start start 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e44067030 0x7f3e44140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44158cc0 0x7f3e4415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e440b5c30 con 0x7f3e44067030 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4415a970 con 0x7f3e44158cc0 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4415a040 con 0x7f3e440cfb00 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:47986/0 (socket says 172.21.15.161:47986) 2024-08-15T16:59:25.960 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 -- 172.21.15.161:0/2722244795 learned_addr learned my addr 172.21.15.161:0/2722244795 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e43fff700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44158cc0 0x7f3e4415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e437fe700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e44067030 0x7f3e44140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 -- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44158cc0 msgr2=0x7f3e4415cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44158cc0 0x7f3e4415cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 -- 172.21.15.161:0/2722244795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e44067030 msgr2=0x7f3e44140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e44067030 0x7f3e44140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 -- 172.21.15.161:0/2722244795 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34009420 con 0x7f3e440cfb00 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e42ffd700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7c0 tx=0x7f3e2c007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c005b30 con 0x7f3e440cfb00 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4415a320 con 0x7f3e440cfb00 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c005cb0 con 0x7f3e440cfb00 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c004a80 con 0x7f3e440cfb00 2024-08-15T16:59:25.961 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e44140f80 con 0x7f3e440cfb00 2024-08-15T16:59:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f3e2c0070f0 con 0x7f3e440cfb00 2024-08-15T16:59:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.943+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f3e2c0053f0 con 0x7f3e440cfb00 2024-08-15T16:59:25.962 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.947+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e440638f0 con 0x7f3e440cfb00 2024-08-15T16:59:25.963 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:25.947+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f3e2c018070 con 0x7f3e440cfb00 2024-08-15T16:59:26.038 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.023+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c36dbee1-7d4f-4553-a18a-c1a70cfc250a", "id": 1} v 0) v1 -- 0x7f3e4415aab0 con 0x7f3e440cfb00 2024-08-15T16:59:26.044 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.027+0000 7f3e3b7fe700 1 -- 172.21.15.161:0/2722244795 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c36dbee1-7d4f-4553-a18a-c1a70cfc250a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3e2c0055b0 con 0x7f3e440cfb00 2024-08-15T16:59:26.045 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-08-15T16:59:26.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 msgr2=0x7f3e4415c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.046 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c7c0 tx=0x7f3e2c007f70 comp rx=0 tx=0).stop 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 shutdown_connections 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e44158cc0 0x7f3e4415cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e440cfb00 0x7f3e4415c890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 --2- 172.21.15.161:0/2722244795 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e44067030 0x7f3e44140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 >> 172.21.15.161:0/2722244795 conn(0x7f3e440d0b50 msgr2=0x7f3e440b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 shutdown_connections 2024-08-15T16:59:26.047 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.031+0000 7f3e4895c700 1 -- 172.21.15.161:0/2722244795 wait complete. 2024-08-15T16:59:26.057 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 2a7a86d9-0c58-408b-b2e1-cce212321635 2 2024-08-15T16:59:26.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.147+0000 7f0fc0910700 1 Processor -- start 2024-08-15T16:59:26.165 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- start start 2024-08-15T16:59:26.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0fbc156240 0x7f0fbc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0fbc0b1740 0x7f0fbc0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc0b1ba0 con 0x7f0fbc155e70 2024-08-15T16:59:26.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0c9b90 con 0x7f0fbc0b1740 2024-08-15T16:59:26.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc0ae120 con 0x7f0fbc156240 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0fbc156240 0x7f0fbc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fba59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0fbc155e70 0x7f0fbc0ada10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58690/0 (socket says 172.21.15.161:58690) 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0fbc156240 0x7f0fbc0b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50000/0 (socket says 172.21.15.161:50000) 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fba59c700 1 -- 172.21.15.161:0/1063330787 learned_addr learned my addr 172.21.15.161:0/1063330787 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fbad9d700 1 --2- 172.21.15.161:0/1063330787 >> v2:172.21.15.146:3300/0 conn(0x7f0fbc0b1740 0x7f0fbc0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb959a700 1 -- 172.21.15.161:0/1063330787 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443421777 0 0) 0x7f0fac001110 con 0x7f0fbc155e70 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 -- 172.21.15.161:0/1063330787 >> v2:172.21.15.146:3300/0 conn(0x7f0fbc0b1740 msgr2=0x7f0fbc0ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 --2- 172.21.15.161:0/1063330787 >> v2:172.21.15.146:3300/0 conn(0x7f0fbc0b1740 0x7f0fbc0ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 -- 172.21.15.161:0/1063330787 >> v1:172.21.15.161:6789/0 conn(0x7f0fbc155e70 legacy=0x7f0fbc0ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 -- 172.21.15.161:0/1063330787 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0ae3d0 con 0x7f0fbc156240 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb9d9b700 1 --2- 172.21.15.161:0/1063330787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc0b1200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0ac900 tx=0x7f0fb00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f805a4dc95c740 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb959a700 1 -- 172.21.15.161:0/1063330787 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0011630 con 0x7f0fbc156240 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb959a700 1 -- 172.21.15.161:0/1063330787 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb0011790 con 0x7f0fbc156240 2024-08-15T16:59:26.168 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fb959a700 1 -- 172.21.15.161:0/1063330787 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0011b30 con 0x7f0fbc156240 2024-08-15T16:59:26.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- 172.21.15.161:0/1063330787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 msgr2=0x7f0fbc0b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 --2- 172.21.15.161:0/1063330787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc0b1200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0ac900 tx=0x7f0fb00091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- 172.21.15.161:0/1063330787 shutdown_connections 2024-08-15T16:59:26.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 --2- 172.21.15.161:0/1063330787 >> v2:172.21.15.146:3300/0 conn(0x7f0fbc0b1740 0x7f0fbc0ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 --2- 172.21.15.161:0/1063330787 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc0b1200 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- 172.21.15.161:0/1063330787 >> 172.21.15.161:0/1063330787 conn(0x7f0fbc0687a0 msgr2=0x7f0fbc06c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- 172.21.15.161:0/1063330787 shutdown_connections 2024-08-15T16:59:26.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.151+0000 7f0fc0910700 1 -- 172.21.15.161:0/1063330787 wait complete. 2024-08-15T16:59:26.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 Processor -- start 2024-08-15T16:59:26.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 -- start start 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 0x7f0fbc149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fbc155e70 0x7f0fbc156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cae10 con 0x7f0fbc155e70 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc14a0d0 con 0x7f0fbc156240 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc0cb0b0 con 0x7f0fbc0b1740 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 0x7f0fbc149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.173 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50006/0 (socket says 172.21.15.161:50006) 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 0x7f0fbc149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:47990/0 (socket says 172.21.15.161:47990) 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 -- 172.21.15.161:0/2415570937 learned_addr learned my addr 172.21.15.161:0/2415570937 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fb9d9b700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fbc155e70 0x7f0fbc156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 -- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 msgr2=0x7f0fbc149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 0x7f0fbc149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 -- 172.21.15.161:0/2415570937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fbc155e70 msgr2=0x7f0fbc156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fbc155e70 0x7f0fbc156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 -- 172.21.15.161:0/2415570937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0009420 con 0x7f0fbc156240 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fbad9d700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009e90 tx=0x7f0fb400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.174 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb400cb70 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb400f040 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc0cb390 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.155+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4014480 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.159+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc0cb950 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.159+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f0fb4014ac0 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.159+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f0fb4014d70 con 0x7f0fbc156240 2024-08-15T16:59:26.175 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.159+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc0645c0 con 0x7f0fbc156240 2024-08-15T16:59:26.178 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.163+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f0fb4018020 con 0x7f0fbc156240 2024-08-15T16:59:26.256 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.239+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a7a86d9-0c58-408b-b2e1-cce212321635", "id": 2} v 0) v1 -- 0x7f0fbc0cbb40 con 0x7f0fbc156240 2024-08-15T16:59:26.258 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.243+0000 7fe5a8472000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T16:59:26.262 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0faa7fc700 1 -- 172.21.15.161:0/2415570937 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a7a86d9-0c58-408b-b2e1-cce212321635", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0fb401d400 con 0x7f0fbc156240 2024-08-15T16:59:26.262 INFO:teuthology.orchestra.run.smithi161.stdout:2 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 msgr2=0x7f0fbc06a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4009e90 tx=0x7f0fb400b040 comp rx=0 tx=0).stop 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 shutdown_connections 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0fbc156240 0x7f0fbc06a730 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0fbc155e70 0x7f0fbc156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 --2- 172.21.15.161:0/2415570937 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0fbc0b1740 0x7f0fbc149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 >> 172.21.15.161:0/2415570937 conn(0x7f0fbc0687a0 msgr2=0x7f0fbc152390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 shutdown_connections 2024-08-15T16:59:26.264 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.247+0000 7f0fc0910700 1 -- 172.21.15.161:0/2415570937 wait complete. 2024-08-15T16:59:26.275 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 7171ad55-1f6b-440e-9ae7-b204c7596597 3 2024-08-15T16:59:26.318 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.303+0000 7fe5a8472000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T16:59:26.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.359+0000 7f955c8aa700 1 Processor -- start 2024-08-15T16:59:26.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.359+0000 7f955c8aa700 1 -- start start 2024-08-15T16:59:26.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 0x7f95580cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9558069800 con 0x7f9558069c90 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95580c89c0 con 0x7f955806a0e0 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95580c8ca0 con 0x7f95580cbc00 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f9556d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 0x7f95580cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:47998/0 (socket says 172.21.15.161:47998) 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 -- 172.21.15.161:0/2093558318 learned_addr learned my addr 172.21.15.161:0/2093558318 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 -- 172.21.15.161:0/2093558318 >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 msgr2=0x7f95580cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 --2- 172.21.15.161:0/2093558318 >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 0x7f95580cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 -- 172.21.15.161:0/2093558318 >> v1:172.21.15.146:6789/0 conn(0x7f9558069c90 legacy=0x7f95580b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 -- 172.21.15.161:0/2093558318 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558069490 con 0x7f95580cbc00 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f9556d9d700 1 --2- 172.21.15.161:0/2093558318 >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 0x7f95580cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955659c700 1 --2- 172.21.15.161:0/2093558318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95480099b0 tx=0x7f9548009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b964098fcb860c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f9555d9b700 1 -- 172.21.15.161:0/2093558318 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548009ea0 con 0x7f95580cbc00 2024-08-15T16:59:26.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f9555d9b700 1 -- 172.21.15.161:0/2093558318 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954800b860 con 0x7f95580cbc00 2024-08-15T16:59:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f9555d9b700 1 -- 172.21.15.161:0/2093558318 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800bc00 con 0x7f95580cbc00 2024-08-15T16:59:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- 172.21.15.161:0/2093558318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 msgr2=0x7f95580ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.380 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 --2- 172.21.15.161:0/2093558318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95480099b0 tx=0x7f9548009250 comp rx=0 tx=0).stop 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- 172.21.15.161:0/2093558318 shutdown_connections 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 --2- 172.21.15.161:0/2093558318 >> v2:172.21.15.161:3300/0 conn(0x7f955806a0e0 0x7f95580cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 --2- 172.21.15.161:0/2093558318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95580cbc00 0x7f95580ca280 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- 172.21.15.161:0/2093558318 >> 172.21.15.161:0/2093558318 conn(0x7f95580caab0 msgr2=0x7f95580caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- 172.21.15.161:0/2093558318 shutdown_connections 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.363+0000 7f955c8aa700 1 -- 172.21.15.161:0/2093558318 wait complete. 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 Processor -- start 2024-08-15T16:59:26.381 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- start start 2024-08-15T16:59:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 0x7f95580a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 0x7f9558157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.384 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95580aacb0 con 0x7f95580cbc00 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95580aa0a0 con 0x7f955806a0e0 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95580aa380 con 0x7f9558069c90 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 0x7f95580a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:48002/0 (socket says 172.21.15.161:48002) 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955659c700 1 -- 172.21.15.161:0/3973765194 learned_addr learned my addr 172.21.15.161:0/3973765194 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f9556d9d700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 0x7f9558157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 -- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 msgr2=0x7f95580c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 -- 172.21.15.161:0/3973765194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 msgr2=0x7f9558157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 0x7f9558157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 -- 172.21.15.161:0/3973765194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009a20 con 0x7f955806a0e0 2024-08-15T16:59:26.385 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f9556d9d700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 0x7f9558157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955659c700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954ffff700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 0x7f95580a9870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f954000c590 tx=0x7f9540007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011590 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9540011710 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95480094f0 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011ad0 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95580aa940 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f954001c440 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.367+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f9540012420 con 0x7f955806a0e0 2024-08-15T16:59:26.386 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.371+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95580645c0 con 0x7f955806a0e0 2024-08-15T16:59:26.387 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.371+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f9540017070 con 0x7f955806a0e0 2024-08-15T16:59:26.437 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.423+0000 7fe5a8472000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T16:59:26.443 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:26.429+0000 7f1ad8ed8000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-15T16:59:26.464 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.447+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7171ad55-1f6b-440e-9ae7-b204c7596597", "id": 3} v 0) v1 -- 0x7f95580ca490 con 0x7f955806a0e0 2024-08-15T16:59:26.471 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.455+0000 7f954e7fc700 1 -- 172.21.15.161:0/3973765194 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7171ad55-1f6b-440e-9ae7-b204c7596597", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f954001c6f0 con 0x7f955806a0e0 2024-08-15T16:59:26.472 INFO:teuthology.orchestra.run.smithi161.stdout:3 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.455+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 msgr2=0x7f95580a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.455+0000 7f955c8aa700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 0x7f95580a9870 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f954000c590 tx=0x7f9540007ec0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.455+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 shutdown_connections 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f95580cbc00 0x7f9558157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f955806a0e0 0x7f95580a9870 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 --2- 172.21.15.161:0/3973765194 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9558069c90 0x7f95580c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 >> 172.21.15.161:0/3973765194 conn(0x7f95580caab0 msgr2=0x7f95580b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.473 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 shutdown_connections 2024-08-15T16:59:26.474 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.459+0000 7f955c8aa700 1 -- 172.21.15.161:0/3973765194 wait complete. 2024-08-15T16:59:26.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new b1084d0b-f46e-4452-b3d7-99809b0c8d1a 4 2024-08-15T16:59:26.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 Processor -- start 2024-08-15T16:59:26.593 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- start start 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0764068db0 0x7f0764159380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07640c8b70 0x7f07640699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076406a3b0 con 0x7f0764158fb0 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0764069ee0 con 0x7f0764068db0 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640c9010 con 0x7f07640c8b70 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07640c8b70 0x7f07640699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0763fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0764158fb0 0x7f07640686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58704/0 (socket says 172.21.15.161:58704) 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0763fff700 1 -- 172.21.15.161:0/12471221 learned_addr learned my addr 172.21.15.161:0/12471221 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:26.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f07637fe700 1 --2- 172.21.15.161:0/12471221 >> v2:172.21.15.146:3300/0 conn(0x7f0764068db0 0x7f0764159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 -- 172.21.15.161:0/12471221 >> v2:172.21.15.146:3300/0 conn(0x7f0764068db0 msgr2=0x7f0764159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 --2- 172.21.15.161:0/12471221 >> v2:172.21.15.146:3300/0 conn(0x7f0764068db0 0x7f0764159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 -- 172.21.15.161:0/12471221 >> v1:172.21.15.161:6789/0 conn(0x7f0764158fb0 legacy=0x7f07640686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 -- 172.21.15.161:0/12471221 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640a9fe0 con 0x7f07640c8b70 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0762ffd700 1 -- 172.21.15.161:0/12471221 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367236874 0 0) 0x7f076406a3b0 con 0x7f0764158fb0 2024-08-15T16:59:26.597 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0768f3a700 1 --2- 172.21.15.161:0/12471221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07640c8b70 0x7f07640699a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f076406ea60 tx=0x7f075c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ecd4b39f501a15 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0762ffd700 1 -- 172.21.15.161:0/12471221 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00e070 con 0x7f07640c8b70 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0762ffd700 1 -- 172.21.15.161:0/12471221 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00bb60 con 0x7f07640c8b70 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f0762ffd700 1 -- 172.21.15.161:0/12471221 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c012430 con 0x7f07640c8b70 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- 172.21.15.161:0/12471221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07640c8b70 msgr2=0x7f07640699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 --2- 172.21.15.161:0/12471221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07640c8b70 0x7f07640699a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f076406ea60 tx=0x7f075c009780 comp rx=0 tx=0).stop 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- 172.21.15.161:0/12471221 shutdown_connections 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 --2- 172.21.15.161:0/12471221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07640c8b70 0x7f07640699a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 --2- 172.21.15.161:0/12471221 >> v2:172.21.15.146:3300/0 conn(0x7f0764068db0 0x7f0764159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- 172.21.15.161:0/12471221 >> 172.21.15.161:0/12471221 conn(0x7f07640c9bf0 msgr2=0x7f076406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- 172.21.15.161:0/12471221 shutdown_connections 2024-08-15T16:59:26.598 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.579+0000 7f076a99d700 1 -- 172.21.15.161:0/12471221 wait complete. 2024-08-15T16:59:26.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 Processor -- start 2024-08-15T16:59:26.599 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 -- start start 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764068db0 0x7f07641402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076415b8a0 con 0x7f0764068db0 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076415a9b0 con 0x7f0764158fb0 2024-08-15T16:59:26.602 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f076a99d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076415ac90 con 0x7f07640c8b70 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f0768f3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f07637fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f0768f3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50046/0 (socket says 172.21.15.161:50046) 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f0768f3a700 1 -- 172.21.15.161:0/3067118893 learned_addr learned my addr 172.21.15.161:0/3067118893 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f07637fe700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:48008/0 (socket says 172.21.15.161:48008) 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f0763fff700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764068db0 0x7f07641402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.583+0000 7f07637fe700 1 -- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 msgr2=0x7f07640b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f07637fe700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f07637fe700 1 -- 172.21.15.161:0/3067118893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764068db0 msgr2=0x7f07641402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f07637fe700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764068db0 0x7f07641402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f07637fe700 1 -- 172.21.15.161:0/3067118893 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009a20 con 0x7f07640c8b70 2024-08-15T16:59:26.603 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f0768f3a700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f07637fe700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f075800e9d0 tx=0x7f07580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758013680 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758013800 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076415af70 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758013bc0 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076415b530 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f0758005c90 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f0758006700 con 0x7f07640c8b70 2024-08-15T16:59:26.604 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.587+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076415ba10 con 0x7f07640c8b70 2024-08-15T16:59:26.606 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.591+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f0758020060 con 0x7f07640c8b70 2024-08-15T16:59:26.665 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.651+0000 7fe5a8472000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T16:59:26.681 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.663+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b1084d0b-f46e-4452-b3d7-99809b0c8d1a", "id": 4} v 0) v1 -- 0x7f07640645c0 con 0x7f07640c8b70 2024-08-15T16:59:26.690 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f074bfff700 1 -- 172.21.15.161:0/3067118893 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b1084d0b-f46e-4452-b3d7-99809b0c8d1a", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0758028020 con 0x7f07640c8b70 2024-08-15T16:59:26.690 INFO:teuthology.orchestra.run.smithi161.stdout:4 2024-08-15T16:59:26.691 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 msgr2=0x7f07640aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f075800e9d0 tx=0x7f07580091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 shutdown_connections 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0764158fb0 0x7f07640b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07640c8b70 0x7f07640aa470 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 --2- 172.21.15.161:0/3067118893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0764068db0 0x7f07641402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 >> 172.21.15.161:0/3067118893 conn(0x7f07640c9bf0 msgr2=0x7f07640b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 shutdown_connections 2024-08-15T16:59:26.692 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.675+0000 7f076a99d700 1 -- 172.21.15.161:0/3067118893 wait complete. 2024-08-15T16:59:26.703 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce 5 2024-08-15T16:59:26.718 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.703+0000 7fe5a8472000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T16:59:26.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 Processor -- start 2024-08-15T16:59:26.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 -- start start 2024-08-15T16:59:26.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8bbc15a680 0x7f8bbc06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc15a1f0 con 0x7f8bbc1570b0 2024-08-15T16:59:26.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc153e00 con 0x7f8bbc153a30 2024-08-15T16:59:26.812 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc256f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc0af0c0 con 0x7f8bbc15a680 2024-08-15T16:59:26.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc0d6c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8bbc15a680 0x7f8bbc06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc156d700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8bbc153a30 0x7f8bbc0ae500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58708/0 (socket says 172.21.15.161:58708) 2024-08-15T16:59:26.813 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bc156d700 1 -- 172.21.15.161:0/1076264961 learned_addr learned my addr 172.21.15.161:0/1076264961 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.795+0000 7f8bb3fff700 1 -- 172.21.15.161:0/1076264961 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430152286 0 0) 0x7f8bbc153e00 con 0x7f8bbc153a30 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc0d6c700 1 -- 172.21.15.161:0/1076264961 >> v1:172.21.15.161:6789/0 conn(0x7f8bbc153a30 legacy=0x7f8bbc0ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc0d6c700 1 -- 172.21.15.161:0/1076264961 >> v1:172.21.15.146:6789/0 conn(0x7f8bbc1570b0 legacy=0x7f8bbc1575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc0d6c700 1 -- 172.21.15.161:0/1076264961 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0aede0 con 0x7f8bbc15a680 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc0d6c700 1 --2- 172.21.15.161:0/1076264961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc06c200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00bb90 tx=0x7f8bac00b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95db38327f227b5b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bb3fff700 1 -- 172.21.15.161:0/1076264961 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00dba0 con 0x7f8bbc15a680 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bb3fff700 1 -- 172.21.15.161:0/1076264961 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00dd00 con 0x7f8bbc15a680 2024-08-15T16:59:26.814 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bb3fff700 1 -- 172.21.15.161:0/1076264961 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0196b0 con 0x7f8bbc15a680 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- 172.21.15.161:0/1076264961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 msgr2=0x7f8bbc06c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 --2- 172.21.15.161:0/1076264961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc06c200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00bb90 tx=0x7f8bac00b430 comp rx=0 tx=0).stop 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- 172.21.15.161:0/1076264961 shutdown_connections 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 --2- 172.21.15.161:0/1076264961 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc06c200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- 172.21.15.161:0/1076264961 >> 172.21.15.161:0/1076264961 conn(0x7f8bbc0ca2b0 msgr2=0x7f8bbc06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.815 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- 172.21.15.161:0/1076264961 shutdown_connections 2024-08-15T16:59:26.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- 172.21.15.161:0/1076264961 wait complete. 2024-08-15T16:59:26.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 Processor -- start 2024-08-15T16:59:26.816 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.799+0000 7f8bc256f700 1 -- start start 2024-08-15T16:59:26.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bbc1570b0 0x7f8bbc06a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:26.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc06a780 con 0x7f8bbc1570b0 2024-08-15T16:59:26.819 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc069db0 con 0x7f8bbc15a680 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc14f960 con 0x7f8bbc153a30 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc1d6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:48016/0 (socket says 172.21.15.161:48016) 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 -- 172.21.15.161:0/2244454047 learned_addr learned my addr 172.21.15.161:0/2244454047 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc1d6e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc14f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50076/0 (socket says 172.21.15.161:50076) 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc0d6c700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bbc1570b0 0x7f8bbc06a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 -- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 msgr2=0x7f8bbc14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 -- 172.21.15.161:0/2244454047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bbc1570b0 msgr2=0x7f8bbc06a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bbc1570b0 0x7f8bbc06a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 -- 172.21.15.161:0/2244454047 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac00b6d0 con 0x7f8bbc153a30 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc156d700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800c840 tx=0x7f8bb8007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:26.820 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8005c00 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb8005d60 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc0691a0 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8007320 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc069760 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f8bb801c440 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f8bb8007710 con 0x7f8bbc153a30 2024-08-15T16:59:26.821 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.803+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc0638f0 con 0x7f8bbc153a30 2024-08-15T16:59:26.823 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.807+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f8bb8017070 con 0x7f8bbc153a30 2024-08-15T16:59:26.891 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.875+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce", "id": 5} v 0) v1 -- 0x7f8bbc155ed0 con 0x7f8bbc153a30 2024-08-15T16:59:26.901 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bb17fa700 1 -- 172.21.15.161:0/2244454047 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8bb801c8d0 con 0x7f8bbc153a30 2024-08-15T16:59:26.901 INFO:teuthology.orchestra.run.smithi161.stdout:5 2024-08-15T16:59:26.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 msgr2=0x7f8bbc06b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:26.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800c840 tx=0x7f8bb8007d90 comp rx=0 tx=0).stop 2024-08-15T16:59:26.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 shutdown_connections 2024-08-15T16:59:26.902 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8bbc15a680 0x7f8bbc14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bbc1570b0 0x7f8bbc06a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 --2- 172.21.15.161:0/2244454047 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8bbc153a30 0x7f8bbc06b150 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:26.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 >> 172.21.15.161:0/2244454047 conn(0x7f8bbc0ca2b0 msgr2=0x7f8bbc150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:26.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 shutdown_connections 2024-08-15T16:59:26.903 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:26.887+0000 7f8bc256f700 1 -- 172.21.15.161:0/2244454047 wait complete. 2024-08-15T16:59:26.911 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.895+0000 7fe5a8472000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T16:59:26.915 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new edc57359-0752-4dfd-a5b2-1ee430f185d7 6 2024-08-15T16:59:26.977 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:26.959+0000 7fe5a8472000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T16:59:27.045 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:27.027+0000 7fe5a8472000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T16:59:27.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.051+0000 7f1146d77700 1 Processor -- start 2024-08-15T16:59:27.068 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.051+0000 7f1146d77700 1 -- start start 2024-08-15T16:59:27.071 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f1140142a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11400c82e0 0x7f11400c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11400b1180 con 0x7f1140142660 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11400c9310 con 0x7f11400c82e0 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11400b0890 con 0x7f11400aec20 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145d75700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1140142660 0x7f11400ae510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:58716/0 (socket says 172.21.15.161:58716) 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145d75700 1 -- 172.21.15.161:0/2966317727 learned_addr learned my addr 172.21.15.161:0/2966317727 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 --2- 172.21.15.161:0/2966317727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f1140142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146576700 1 --2- 172.21.15.161:0/2966317727 >> v2:172.21.15.146:3300/0 conn(0x7f11400c82e0 0x7f11400c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1144d73700 1 -- 172.21.15.161:0/2966317727 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637707744 0 0) 0x7f11400b1180 con 0x7f1140142660 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1144d73700 1 -- 172.21.15.161:0/2966317727 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f112c003570 con 0x7f1140142660 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 -- 172.21.15.161:0/2966317727 >> v2:172.21.15.146:3300/0 conn(0x7f11400c82e0 msgr2=0x7f11400c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:27.072 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 --2- 172.21.15.161:0/2966317727 >> v2:172.21.15.146:3300/0 conn(0x7f11400c82e0 0x7f11400c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 -- 172.21.15.161:0/2966317727 >> v1:172.21.15.161:6789/0 conn(0x7f1140142660 legacy=0x7f11400ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 -- 172.21.15.161:0/2966317727 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400c9030 con 0x7f11400aec20 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1145574700 1 --2- 172.21.15.161:0/2966317727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f1140142a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f11300098e0 tx=0x7f11300091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9b7f72832a9a4d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1144d73700 1 -- 172.21.15.161:0/2966317727 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113000b640 con 0x7f11400aec20 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1144d73700 1 -- 172.21.15.161:0/2966317727 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113000b7a0 con 0x7f11400aec20 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1144d73700 1 -- 172.21.15.161:0/2966317727 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113000bb40 con 0x7f11400aec20 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- 172.21.15.161:0/2966317727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 msgr2=0x7f1140142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 --2- 172.21.15.161:0/2966317727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f1140142a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f11300098e0 tx=0x7f11300091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- 172.21.15.161:0/2966317727 shutdown_connections 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 --2- 172.21.15.161:0/2966317727 >> v2:172.21.15.146:3300/0 conn(0x7f11400c82e0 0x7f11400c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 --2- 172.21.15.161:0/2966317727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f1140142a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- 172.21.15.161:0/2966317727 >> 172.21.15.161:0/2966317727 conn(0x7f11400c9830 msgr2=0x7f11400c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- 172.21.15.161:0/2966317727 shutdown_connections 2024-08-15T16:59:27.073 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.055+0000 7f1146d77700 1 -- 172.21.15.161:0/2966317727 wait complete. 2024-08-15T16:59:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 Processor -- start 2024-08-15T16:59:27.074 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- start start 2024-08-15T16:59:27.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f11400c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.077 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 0x7f114012fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401309e0 con 0x7f1140142660 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140130c80 con 0x7f11400c82e0 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140130f60 con 0x7f11400aec20 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145d75700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f11400c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50088/0 (socket says 172.21.15.161:50088) 2024-08-15T16:59:27.078 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 -- 172.21.15.161:0/615512608 learned_addr learned my addr 172.21.15.161:0/615512608 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146576700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 0x7f114012fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 -- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 msgr2=0x7f11400c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f11400c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 -- 172.21.15.161:0/615512608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 msgr2=0x7f114012fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 0x7f114012fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 -- 172.21.15.161:0/615512608 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130009420 con 0x7f11400c82e0 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146576700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 0x7f114012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1145574700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f11400b09d0 tx=0x7f1130017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113001bcf0 con 0x7f11400c82e0 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113001be50 con 0x7f11400c82e0 2024-08-15T16:59:27.079 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1140131240 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130011670 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.059+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1140130310 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.063+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f11300119e0 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.063+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f1130025d20 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.063+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1140131450 con 0x7f11400c82e0 2024-08-15T16:59:27.080 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.063+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f113002a060 con 0x7f11400c82e0 2024-08-15T16:59:27.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.143+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "edc57359-0752-4dfd-a5b2-1ee430f185d7", "id": 6} v 0) v1 -- 0x7f11400638f0 con 0x7f11400c82e0 2024-08-15T16:59:27.167 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1135ffb700 1 -- 172.21.15.161:0/615512608 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "edc57359-0752-4dfd-a5b2-1ee430f185d7", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1130015020 con 0x7f11400c82e0 2024-08-15T16:59:27.167 INFO:teuthology.orchestra.run.smithi161.stdout:6 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 msgr2=0x7f114012f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f11400b09d0 tx=0x7f1130017000 comp rx=0 tx=0).stop 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 shutdown_connections 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1140142660 0x7f114012fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11400c82e0 0x7f114012f5a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 --2- 172.21.15.161:0/615512608 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11400aec20 0x7f11400c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 >> 172.21.15.161:0/615512608 conn(0x7f11400c9830 msgr2=0x7f1140141750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.169 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.151+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 shutdown_connections 2024-08-15T16:59:27.170 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.155+0000 7f1146d77700 1 -- 172.21.15.161:0/615512608 wait complete. 2024-08-15T16:59:27.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 4fb88d84-7e68-48c6-abac-c014247e680e 7 2024-08-15T16:59:27.195 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.181+0000 7f1ad8ed8000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-15T16:59:27.252 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.241+0000 7f1ad8ed8000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-15T16:59:27.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 Processor -- start 2024-08-15T16:59:27.281 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- start start 2024-08-15T16:59:27.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 0x7f63a0069930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f63a0156000 0x7f63a0069e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a013be40 con 0x7f63a0155c30 2024-08-15T16:59:27.283 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a013b230 con 0x7f63a0156000 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a013b510 con 0x7f63a013a4c0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:48046/0 (socket says 172.21.15.161:48046) 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 -- 172.21.15.161:0/2479939537 learned_addr learned my addr 172.21.15.161:0/2479939537 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639f7fe700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.161:3300/0 conn(0x7f63a0156000 0x7f63a0069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639ffff700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 0x7f63a0069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 -- 172.21.15.161:0/2479939537 >> v2:172.21.15.161:3300/0 conn(0x7f63a0156000 msgr2=0x7f63a0069e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.161:3300/0 conn(0x7f63a0156000 0x7f63a0069e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 -- 172.21.15.161:0/2479939537 >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 msgr2=0x7f63a0069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 0x7f63a0069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 -- 172.21.15.161:0/2479939537 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a013b7f0 con 0x7f63a013a4c0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639ffff700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 0x7f63a0069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a4dcd700 1 --2- 172.21.15.161:0/2479939537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63a006d5c0 tx=0x7f6398009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733aa6b75245d5fc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.284 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639effd700 1 -- 172.21.15.161:0/2479939537 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800e070 con 0x7f63a013a4c0 2024-08-15T16:59:27.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639effd700 1 -- 172.21.15.161:0/2479939537 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639800ba20 con 0x7f63a013a4c0 2024-08-15T16:59:27.285 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f639effd700 1 -- 172.21.15.161:0/2479939537 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63980092e0 con 0x7f63a013a4c0 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- 172.21.15.161:0/2479939537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 msgr2=0x7f63a013a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- 172.21.15.161:0/2479939537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63a006d5c0 tx=0x7f6398009840 comp rx=0 tx=0).stop 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- 172.21.15.161:0/2479939537 shutdown_connections 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- 172.21.15.161:0/2479939537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a013a4c0 0x7f63a013a920 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.161:3300/0 conn(0x7f63a0156000 0x7f63a0069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 --2- 172.21.15.161:0/2479939537 >> v2:172.21.15.146:3300/0 conn(0x7f63a0155c30 0x7f63a0069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- 172.21.15.161:0/2479939537 >> 172.21.15.161:0/2479939537 conn(0x7f63a006e170 msgr2=0x7f63a006e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- 172.21.15.161:0/2479939537 shutdown_connections 2024-08-15T16:59:27.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.267+0000 7f63a55ce700 1 -- 172.21.15.161:0/2479939537 wait complete. 2024-08-15T16:59:27.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 Processor -- start 2024-08-15T16:59:27.287 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 -- start start 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a013a4c0 0x7f63a0156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a0156000 0x7f63a00ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00a9d30 con 0x7f63a013a4c0 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00aa010 con 0x7f63a0155c30 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a55ce700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0156c50 con 0x7f63a0156000 2024-08-15T16:59:27.289 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f63a4dcd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a0156000 0x7f63a00ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:50108/0 (socket says 172.21.15.161:50108) 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 -- 172.21.15.161:0/3414175319 learned_addr learned my addr 172.21.15.161:0/3414175319 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639ffff700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a013a4c0 0x7f63a0156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 -- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a0156000 msgr2=0x7f63a00ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a0156000 0x7f63a00ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 -- 172.21.15.161:0/3414175319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a013a4c0 msgr2=0x7f63a0156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a013a4c0 0x7f63a0156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 -- 172.21.15.161:0/3414175319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398009ae0 con 0x7f63a0155c30 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f639f7fe700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f639400e950 tx=0x7f63940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394013680 con 0x7f63a0155c30 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940137e0 con 0x7f63a0155c30 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.271+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394013ba0 con 0x7f63a0155c30 2024-08-15T16:59:27.290 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.275+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0156f30 con 0x7f63a0155c30 2024-08-15T16:59:27.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.275+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0156940 con 0x7f63a0155c30 2024-08-15T16:59:27.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.275+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6394005a00 con 0x7f63a0155c30 2024-08-15T16:59:27.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.275+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f639400cb00 con 0x7f63a0155c30 2024-08-15T16:59:27.291 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.275+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a00aa180 con 0x7f63a0155c30 2024-08-15T16:59:27.293 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.279+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f6394021060 con 0x7f63a0155c30 2024-08-15T16:59:27.367 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.351+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4fb88d84-7e68-48c6-abac-c014247e680e", "id": 7} v 0) v1 -- 0x7f63a00638f0 con 0x7f63a0155c30 2024-08-15T16:59:27.369 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.357+0000 7f1ad8ed8000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-15T16:59:27.377 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f6387fff700 1 -- 172.21.15.161:0/3414175319 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4fb88d84-7e68-48c6-abac-c014247e680e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6394017070 con 0x7f63a0155c30 2024-08-15T16:59:27.377 INFO:teuthology.orchestra.run.smithi161.stdout:7 2024-08-15T16:59:27.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 msgr2=0x7f63a00cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.378 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f639400e950 tx=0x7f63940091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 shutdown_connections 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63a0156000 0x7f63a00ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63a0155c30 0x7f63a00cc4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 --2- 172.21.15.161:0/3414175319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63a013a4c0 0x7f63a0156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 >> 172.21.15.161:0/3414175319 conn(0x7f63a006e170 msgr2=0x7f63a0152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 shutdown_connections 2024-08-15T16:59:27.379 INFO:teuthology.orchestra.run.smithi161.stderr:2024-08-15T16:59:27.363+0000 7f63a55ce700 1 -- 172.21.15.161:0/3414175319 wait complete. 2024-08-15T16:59:27.390 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-15T16:59:27.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-15T16:59:27.393 INFO:tasks.ceph.osd.0:Started 2024-08-15T16:59:27.393 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-15T16:59:27.393 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-15T16:59:27.395 INFO:tasks.ceph.osd.1:Started 2024-08-15T16:59:27.395 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-15T16:59:27.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-15T16:59:27.397 INFO:tasks.ceph.osd.2:Started 2024-08-15T16:59:27.398 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-15T16:59:27.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-15T16:59:27.400 INFO:tasks.ceph.osd.3:Started 2024-08-15T16:59:27.400 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-15T16:59:27.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-15T16:59:27.402 INFO:tasks.ceph.osd.4:Started 2024-08-15T16:59:27.402 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-15T16:59:27.403 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-15T16:59:27.405 INFO:tasks.ceph.osd.5:Started 2024-08-15T16:59:27.405 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-15T16:59:27.405 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-15T16:59:27.407 INFO:tasks.ceph.osd.6:Started 2024-08-15T16:59:27.407 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-15T16:59:27.407 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-15T16:59:27.409 INFO:tasks.ceph.osd.7:Started 2024-08-15T16:59:27.409 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:27.497 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:27.479+0000 7fe5a8472000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T16:59:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.537+0000 7f1c7ffff700 1 Processor -- start 2024-08-15T16:59:27.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.537+0000 7f1c7ffff700 1 -- start start 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1c80153020 0x7f1c800b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c80156a00 con 0x7f1c80068db0 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c80155b30 con 0x7f1c800b2290 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c80155df0 con 0x7f1c80153020 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7f7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1c80068db0 0x7f1c800ad810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34608/0 (socket says 172.21.15.146:34608) 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7f7fe700 1 -- 172.21.15.146:0/555598935 learned_addr learned my addr 172.21.15.146:0/555598935 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7effd700 1 --2- 172.21.15.146:0/555598935 >> v2:172.21.15.161:3300/0 conn(0x7f1c80153020 0x7f1c800b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7dffb700 1 -- 172.21.15.146:0/555598935 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243323035 0 0) 0x7f1c80156a00 con 0x7f1c80068db0 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7dffb700 1 -- 172.21.15.146:0/555598935 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c68003570 con 0x7f1c80068db0 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7effd700 1 -- 172.21.15.146:0/555598935 >> v1:172.21.15.161:6789/0 conn(0x7f1c800b2290 legacy=0x7f1c800686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7effd700 1 -- 172.21.15.146:0/555598935 >> v1:172.21.15.146:6789/0 conn(0x7f1c80068db0 legacy=0x7f1c800ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7effd700 1 -- 172.21.15.146:0/555598935 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c801560d0 con 0x7f1c80153020 2024-08-15T16:59:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7effd700 1 --2- 172.21.15.146:0/555598935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80153020 0x7f1c800b1d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000aa90 tx=0x7f1c7000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2787b36bde38728 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7dffb700 1 -- 172.21.15.146:0/555598935 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000c970 con 0x7f1c80153020 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7dffb700 1 -- 172.21.15.146:0/555598935 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7000cad0 con 0x7f1c80153020 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7dffb700 1 -- 172.21.15.146:0/555598935 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7000ce70 con 0x7f1c80153020 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- 172.21.15.146:0/555598935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80153020 msgr2=0x7f1c800b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/555598935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80153020 0x7f1c800b1d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000aa90 tx=0x7f1c7000a330 comp rx=0 tx=0).stop 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- 172.21.15.146:0/555598935 shutdown_connections 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/555598935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80153020 0x7f1c800b1d50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- 172.21.15.146:0/555598935 >> 172.21.15.146:0/555598935 conn(0x7f1c8006d870 msgr2=0x7f1c8006dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- 172.21.15.146:0/555598935 shutdown_connections 2024-08-15T16:59:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- 172.21.15.146:0/555598935 wait complete. 2024-08-15T16:59:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 Processor -- start 2024-08-15T16:59:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.541+0000 7f1c7ffff700 1 -- start start 2024-08-15T16:59:27.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c800b2290 0x7f1c8013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 0x7f1c80154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8016a700 con 0x7f1c80153020 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80169b80 con 0x7f1c80068db0 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80169e60 con 0x7f1c800b2290 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 0x7f1c80154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 0x7f1c80154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:43812/0 (socket says 172.21.15.146:43812) 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:47678/0 (socket says 172.21.15.146:47678) 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7f7fe700 1 -- 172.21.15.146:0/3679536748 learned_addr learned my addr 172.21.15.146:0/3679536748 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7e7fc700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c800b2290 0x7f1c8013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 -- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c800b2290 msgr2=0x7f1c8013f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c800b2290 0x7f1c8013f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 -- 172.21.15.146:0/3679536748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 msgr2=0x7f1c80154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 0x7f1c80154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 -- 172.21.15.146:0/3679536748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7000a5d0 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7effd700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c70017000 tx=0x7f1c7001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70021430 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c70021590 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8016a140 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70021950 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8016dae0 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1c7002a3f0 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f1c70010ad0 con 0x7f1c80068db0 2024-08-15T16:59:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.545+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c800638f0 con 0x7f1c80068db0 2024-08-15T16:59:27.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.553+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f1c70026070 con 0x7f1c80068db0 2024-08-15T16:59:27.616 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:27.599+0000 7fe5a8472000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T16:59:27.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.617+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c800695b0 con 0x7f1c80068db0 2024-08-15T16:59:27.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.617+0000 7f1c66ffd700 1 -- 172.21.15.146:0/3679536748 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f1c7000b040 con 0x7f1c80068db0 2024-08-15T16:59:27.632 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:27.632 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":10,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:27.358892+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 msgr2=0x7f1c8016d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c70017000 tx=0x7f1c7001a000 comp rx=0 tx=0).stop 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 shutdown_connections 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c80153020 0x7f1c80154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c800b2290 0x7f1c8013f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 --2- 172.21.15.146:0/3679536748 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c80068db0 0x7f1c8016d5a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 >> 172.21.15.146:0/3679536748 conn(0x7f1c8006d870 msgr2=0x7f1c801533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 shutdown_connections 2024-08-15T16:59:27.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:27.621+0000 7f1c7ffff700 1 -- 172.21.15.146:0/3679536748 wait complete. 2024-08-15T16:59:27.643 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-15T16:59:27.644 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-15T16:59:27.651 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.637+0000 7f1ad8ed8000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-15T16:59:27.700 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.685+0000 7f1ad8ed8000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-15T16:59:27.762 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-08-15T16:59:27.749+0000 7efd2cfcf3c0 -1 Falling back to public interface 2024-08-15T16:59:27.762 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-08-15T16:59:27.749+0000 7ffa5e5253c0 -1 Falling back to public interface 2024-08-15T16:59:27.778 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-08-15T16:59:27.763+0000 7f28e60cd3c0 -1 Falling back to public interface 2024-08-15T16:59:27.789 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-08-15T16:59:27.777+0000 7f15644823c0 -1 Falling back to public interface 2024-08-15T16:59:27.794 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-08-15T16:59:27.779+0000 7f13bd2293c0 -1 Falling back to public interface 2024-08-15T16:59:27.868 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.853+0000 7f1ad8ed8000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-15T16:59:27.925 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.913+0000 7f1ad8ed8000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-15T16:59:27.935 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:27.919+0000 7fe5a8472000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T16:59:27.986 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:27.973+0000 7f1ad8ed8000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-15T16:59:28.029 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-08-15T16:59:28.017+0000 7f56656f63c0 -1 Falling back to public interface 2024-08-15T16:59:28.034 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-08-15T16:59:28.019+0000 7f0f5f37f3c0 -1 Falling back to public interface 2024-08-15T16:59:28.043 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-08-15T16:59:28.027+0000 7ff7a84693c0 -1 Falling back to public interface 2024-08-15T16:59:28.101 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.083+0000 7fe5a8472000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T16:59:28.167 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.151+0000 7fe5a8472000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T16:59:28.219 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.203+0000 7fe5a8472000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T16:59:28.276 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.259+0000 7fe5a8472000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T16:59:28.389 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:28.377+0000 7f1ad8ed8000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-15T16:59:28.448 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:28.454 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.439+0000 7fe5a8472000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T16:59:28.503 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:28.489+0000 7f1ad8ed8000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-15T16:59:28.545 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 Processor -- start 2024-08-15T16:59:28.545 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 -- start start 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f707c13f830 0x7f707c140220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f707c06adc0 0x7f707c0c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707c0c8ec0 con 0x7f707c13fc00 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0c91a0 con 0x7f707c06adc0 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f708241f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f707c0c9480 con 0x7f707c13f830 2024-08-15T16:59:28.546 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f707c06adc0 0x7f707c0c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:28.547 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f707c06adc0 0x7f707c0c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60868/0 (socket says 172.21.15.146:60868) 2024-08-15T16:59:28.547 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 -- 172.21.15.146:0/3707387907 learned_addr learned my addr 172.21.15.146:0/3707387907 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:28.547 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f707bfff700 1 --2- 172.21.15.146:0/3707387907 >> v2:172.21.15.161:3300/0 conn(0x7f707c13f830 0x7f707c140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 -- 172.21.15.146:0/3707387907 >> v2:172.21.15.161:3300/0 conn(0x7f707c13f830 msgr2=0x7f707c140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 --2- 172.21.15.146:0/3707387907 >> v2:172.21.15.161:3300/0 conn(0x7f707c13f830 0x7f707c140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 -- 172.21.15.146:0/3707387907 >> v1:172.21.15.161:6789/0 conn(0x7f707c13fc00 legacy=0x7f707c06a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 -- 172.21.15.146:0/3707387907 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0c63c0 con 0x7f707c06adc0 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f70809bc700 1 --2- 172.21.15.146:0/3707387907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c06adc0 0x7f707c0c6bc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7074009f20 tx=0x7f7074009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5aaeb0899b0a534 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f707b7fe700 1 -- 172.21.15.146:0/3707387907 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400e070 con 0x7f707c06adc0 2024-08-15T16:59:28.548 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f707b7fe700 1 -- 172.21.15.146:0/3707387907 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707400ba40 con 0x7f707c06adc0 2024-08-15T16:59:28.549 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.533+0000 7f707b7fe700 1 -- 172.21.15.146:0/3707387907 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740092e0 con 0x7f707c06adc0 2024-08-15T16:59:28.549 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- 172.21.15.146:0/3707387907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c06adc0 msgr2=0x7f707c0c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:28.549 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 --2- 172.21.15.146:0/3707387907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c06adc0 0x7f707c0c6bc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7074009f20 tx=0x7f7074009840 comp rx=0 tx=0).stop 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- 172.21.15.146:0/3707387907 shutdown_connections 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 --2- 172.21.15.146:0/3707387907 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c06adc0 0x7f707c0c6bc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 --2- 172.21.15.146:0/3707387907 >> v2:172.21.15.161:3300/0 conn(0x7f707c13f830 0x7f707c140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- 172.21.15.146:0/3707387907 >> 172.21.15.146:0/3707387907 conn(0x7f707c0ccfa0 msgr2=0x7f707c0cd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- 172.21.15.146:0/3707387907 shutdown_connections 2024-08-15T16:59:28.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- 172.21.15.146:0/3707387907 wait complete. 2024-08-15T16:59:28.551 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 Processor -- start 2024-08-15T16:59:28.551 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.537+0000 7f708241f700 1 -- start start 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f707c06adc0 0x7f707c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f707c13fc00 0x7f707c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1594c0 con 0x7f707c13f830 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c15b970 con 0x7f707c13fc00 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c15b4e0 con 0x7f707c06adc0 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60876/0 (socket says 172.21.15.146:60876) 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 -- 172.21.15.146:0/1988274172 learned_addr learned my addr 172.21.15.146:0/1988274172 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:28.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f70809bc700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f707c13fc00 0x7f707c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:28.554 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f707bfff700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f707c06adc0 0x7f707c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 -- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f707c06adc0 msgr2=0x7f707c0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f707c06adc0 0x7f707c0b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 -- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f707c13fc00 msgr2=0x7f707c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f707c13fc00 0x7f707c158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 -- 172.21.15.146:0/1988274172 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7074009ae0 con 0x7f707c13f830 2024-08-15T16:59:28.555 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7073fff700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f706400c510 tx=0x7f706400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064011590 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064011710 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c15a8d0 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064011ad0 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c15af10 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f706401c440 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f7064012b00 con 0x7f707c13f830 2024-08-15T16:59:28.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.541+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c064c10 con 0x7f707c13f830 2024-08-15T16:59:28.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.545+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7064017070 con 0x7f707c13f830 2024-08-15T16:59:28.621 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.603+0000 7fe5a8472000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T16:59:28.623 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.609+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f707c15b620 con 0x7f707c13f830 2024-08-15T16:59:28.624 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f7078ff9700 1 -- 172.21.15.146:0/1988274172 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f706401c8d0 con 0x7f707c13f830 2024-08-15T16:59:28.624 INFO:teuthology.misc.health.smithi146.stdout: 2024-08-15T16:59:28.624 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":10,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:27.358892+0000","last_up_change":"0.000000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:28.626 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 msgr2=0x7f707c158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:28.626 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f706400c510 tx=0x7f706400c820 comp rx=0 tx=0).stop 2024-08-15T16:59:28.626 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 shutdown_connections 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f707c13fc00 0x7f707c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f707c13f830 0x7f707c158a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 --2- 172.21.15.146:0/1988274172 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f707c06adc0 0x7f707c0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 >> 172.21.15.146:0/1988274172 conn(0x7f707c0ccfa0 msgr2=0x7f707c062fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 shutdown_connections 2024-08-15T16:59:28.627 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:28.613+0000 7f708241f700 1 -- 172.21.15.146:0/1988274172 wait complete. 2024-08-15T16:59:28.639 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-15T16:59:28.686 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.671+0000 7fe5a8472000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T16:59:28.800 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:28.789+0000 7f1ad8ed8000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-15T16:59:28.846 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.831+0000 7fe5a8472000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T16:59:28.911 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:28.895+0000 7fe5a8472000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T16:59:28.965 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:28.953+0000 7f1ad8ed8000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-15T16:59:29.031 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.017+0000 7f1ad8ed8000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-15T16:59:29.083 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.069+0000 7f1ad8ed8000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-15T16:59:29.142 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.129+0000 7f1ad8ed8000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-15T16:59:29.321 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.309+0000 7f1ad8ed8000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-15T16:59:29.489 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.477+0000 7f1ad8ed8000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-15T16:59:29.556 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.541+0000 7f1ad8ed8000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-15T16:59:29.619 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:29.603+0000 7fe5a8472000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T16:59:29.721 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.709+0000 7f1ad8ed8000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-15T16:59:29.789 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:29.777+0000 7f1ad8ed8000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-15T16:59:30.520 INFO:tasks.ceph.mgr.y.smithi146.stderr:2024-08-15T16:59:30.505+0000 7f1ad8ed8000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-15T16:59:30.663 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:30.647+0000 7fe5914e3700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-15T16:59:30.664 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-08-15T16:59:30.647+0000 7fe5914e3700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-15T16:59:31.343 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-08-15T16:59:31.329+0000 7ffa5e5253c0 -1 osd.1 0 log_to_monitors true 2024-08-15T16:59:31.366 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-08-15T16:59:31.351+0000 7f13bd2293c0 -1 osd.7 0 log_to_monitors true 2024-08-15T16:59:31.556 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-08-15T16:59:31.545+0000 7efd2cfcf3c0 -1 osd.3 0 log_to_monitors true 2024-08-15T16:59:31.612 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-08-15T16:59:31.601+0000 7f15644823c0 -1 osd.2 0 log_to_monitors true 2024-08-15T16:59:31.638 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-08-15T16:59:31.623+0000 7f28e60cd3c0 -1 osd.6 0 log_to_monitors true 2024-08-15T16:59:31.853 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-08-15T16:59:31.840+0000 7ff7a84693c0 -1 osd.4 0 log_to_monitors true 2024-08-15T16:59:31.882 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-08-15T16:59:31.869+0000 7f56656f63c0 -1 osd.0 0 log_to_monitors true 2024-08-15T16:59:31.895 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-08-15T16:59:31.880+0000 7f0f5f37f3c0 -1 osd.5 0 log_to_monitors true 2024-08-15T16:59:34.362 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-08-15T16:59:34.348+0000 7f28e186d700 -1 osd.6 0 waiting for initial osdmap 2024-08-15T16:59:34.366 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-08-15T16:59:34.352+0000 7f28da610700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:34.391 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-08-15T16:59:34.376+0000 7f13b89c9700 -1 osd.7 0 waiting for initial osdmap 2024-08-15T16:59:34.413 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-08-15T16:59:34.400+0000 7f13b176c700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:34.593 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-08-15T16:59:34.581+0000 7f155fc22700 -1 osd.2 0 waiting for initial osdmap 2024-08-15T16:59:34.599 INFO:tasks.ceph.osd.2.smithi146.stderr:2024-08-15T16:59:34.585+0000 7f15589c5700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:34.654 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-08-15T16:59:34.641+0000 7ffa59cc5700 -1 osd.1 0 waiting for initial osdmap 2024-08-15T16:59:34.663 INFO:tasks.ceph.osd.1.smithi146.stderr:2024-08-15T16:59:34.653+0000 7ffa52a68700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:34.669 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-08-15T16:59:34.657+0000 7efd29f94700 -1 osd.3 0 waiting for initial osdmap 2024-08-15T16:59:34.676 INFO:tasks.ceph.osd.3.smithi146.stderr:2024-08-15T16:59:34.665+0000 7efd21512700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:35.092 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-08-15T16:59:35.081+0000 7f56626bb700 -1 osd.0 0 waiting for initial osdmap 2024-08-15T16:59:35.101 INFO:tasks.ceph.osd.0.smithi146.stderr:2024-08-15T16:59:35.089+0000 7f5659c39700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:35.283 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-08-15T16:59:35.268+0000 7f0f5c344700 -1 osd.5 0 waiting for initial osdmap 2024-08-15T16:59:35.289 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-08-15T16:59:35.272+0000 7f0f538c2700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:35.293 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-08-15T16:59:35.280+0000 7ff7a3c09700 -1 osd.4 0 waiting for initial osdmap 2024-08-15T16:59:35.298 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-08-15T16:59:35.284+0000 7ff79c9ac700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-15T16:59:35.447 DEBUG:teuthology.orchestra.run.smithi146:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:35.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.537+0000 7f0156567700 1 Processor -- start 2024-08-15T16:59:35.550 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.537+0000 7f0156567700 1 -- start start 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0156567700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f015006c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0156567700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015006c890 con 0x7f015013b4b0 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0156567700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015006cb30 con 0x7f01500c3570 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0156567700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015013bc50 con 0x7f015013b880 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0155565700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f015013b4b0 0x7f01500c2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56146/0 (socket says 172.21.15.146:56146) 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0155565700 1 -- 172.21.15.146:0/1620015865 learned_addr learned my addr 172.21.15.146:0/1620015865 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f015006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314158860 0 0) 0x7f015006c890 con 0x7f015013b4b0 2024-08-15T16:59:35.553 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f013c003570 con 0x7f015013b4b0 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083005531 0 0) 0x7f013c003570 con 0x7f015013b4b0 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 msgr2=0x7f015006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f015006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 >> v1:172.21.15.161:6789/0 conn(0x7f01500c3570 legacy=0x7f015013c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015006cdd0 con 0x7f015013b4b0 2024-08-15T16:59:35.556 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f014c001470 con 0x7f015013b4b0 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f015006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 >> v1:172.21.15.146:6789/0 conn(0x7f015013b4b0 legacy=0x7f01500c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013c003a90 0x7f013c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f013c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f013c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015013bc50 con 0x7f013c004810 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013c00bb70 con 0x7f013c003a90 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013c00bd30 con 0x7f015013b880 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f013c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0155565700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013c003a90 0x7f013c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0155d66700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f013c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.557 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 -- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 msgr2=0x7f013c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015013b880 0x7f013c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 -- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013c003a90 msgr2=0x7f013c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f013c003a90 0x7f013c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 -- 172.21.15.146:0/1620015865 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009420 con 0x7f013c004810 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0154d64700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f013c004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0140009df0 tx=0x7f01400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d238d449d9e989 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140009740 con 0x7f013c004810 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014000e920 con 0x7f013c004810 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.541+0000 7f0147fff700 1 -- 172.21.15.146:0/1620015865 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014000ece0 con 0x7f013c004810 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- 172.21.15.146:0/1620015865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 msgr2=0x7f013c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 --2- 172.21.15.146:0/1620015865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f013c004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0140009df0 tx=0x7f01400091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155d66700 1 -- 172.21.15.146:0/1620015865 reap_dead start 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- 172.21.15.146:0/1620015865 shutdown_connections 2024-08-15T16:59:35.558 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- 172.21.15.146:0/1620015865 >> 172.21.15.146:0/1620015865 conn(0x7f01500672b0 msgr2=0x7f0150067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:35.559 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- 172.21.15.146:0/1620015865 shutdown_connections 2024-08-15T16:59:35.559 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- 172.21.15.146:0/1620015865 wait complete. 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 Processor -- start 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- start start 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0150165590 0x7f0150165a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 0x7f0150157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500c6610 con 0x7f013c004810 2024-08-15T16:59:35.561 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500c7190 con 0x7f0150156b90 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0156567700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150155be0 con 0x7f0150165590 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60938/0 (socket says 172.21.15.146:60938) 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 -- 172.21.15.146:0/2888694507 learned_addr learned my addr 172.21.15.146:0/2888694507 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0154d64700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0150165590 0x7f0150165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155d66700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 0x7f0150157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 -- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0150165590 msgr2=0x7f0150165a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0150165590 0x7f0150165a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 -- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 msgr2=0x7f0150157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 0x7f0150157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 -- 172.21.15.146:0/2888694507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150137ae0 con 0x7f013c004810 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155d66700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 0x7f0150157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:35.562 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f0155565700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f014c007a10 tx=0x7f014c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.545+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c005be0 con 0x7f013c004810 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c005d40 con 0x7f013c004810 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01500c6860 con 0x7f013c004810 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00db90 con 0x7f013c004810 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01500c6e20 con 0x7f013c004810 2024-08-15T16:59:35.563 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01500c6fe0 con 0x7f013c004810 2024-08-15T16:59:35.564 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f014c004830 con 0x7f013c004810 2024-08-15T16:59:35.564 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f01457fa700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 0x7f0134062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.564 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f014c07c4d0 con 0x7f013c004810 2024-08-15T16:59:35.564 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f0154d64700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 0x7f0134062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.564 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.549+0000 7f0154d64700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 0x7f0134062c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f01400093f0 tx=0x7f014001a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.569 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.557+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f014c04ced0 con 0x7f013c004810 2024-08-15T16:59:35.737 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.725+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0150067740 con 0x7f013c004810 2024-08-15T16:59:35.738 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.725+0000 7f01457fa700 1 -- 172.21.15.146:0/2888694507 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10444 (secure 0 0 0) 0x7f0150067740 con 0x7f013c004810 2024-08-15T16:59:35.738 INFO:teuthology.misc.health.smithi146.stdout: 2024-08-15T16:59:35.738 INFO:teuthology.misc.health.smithi146.stdout:{"epoch":14,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:35.356214+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:35.740 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 msgr2=0x7f0134062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.740 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 0x7f0134062c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f01400093f0 tx=0x7f014001a000 comp rx=0 tx=0).stop 2024-08-15T16:59:35.740 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 msgr2=0x7f01501375a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.740 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f014c007a10 tx=0x7f014c009040 comp rx=0 tx=0).stop 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 shutdown_connections 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f01340607d0 0x7f0134062c80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150156b90 0x7f0150157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0150165590 0x7f0150165a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 --2- 172.21.15.146:0/2888694507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f013c004810 0x7f01501375a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 >> 172.21.15.146:0/2888694507 conn(0x7f01500672b0 msgr2=0x7f0150067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 shutdown_connections 2024-08-15T16:59:35.741 INFO:teuthology.misc.health.smithi146.stderr:2024-08-15T16:59:35.729+0000 7f0156567700 1 -- 172.21.15.146:0/2888694507 wait complete. 2024-08-15T16:59:35.750 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-15T16:59:35.750 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-15T16:59:35.751 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T16:59:35.751 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T16:59:35.753 INFO:tasks.ceph.mds.a:Started 2024-08-15T16:59:35.753 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T16:59:35.753 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T16:59:35.755 INFO:tasks.ceph.mds.c:Started 2024-08-15T16:59:35.756 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T16:59:35.756 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T16:59:35.759 INFO:tasks.ceph.mds.b:Started 2024-08-15T16:59:35.759 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T16:59:35.759 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T16:59:35.761 INFO:tasks.ceph.mds.d:Started 2024-08-15T16:59:35.761 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-15T16:59:35.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:35.817 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-08-15T16:59:35.824 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-08-15T16:59:35.832 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-08-15T16:59:35.839 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-08-15T16:59:35.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 Processor -- start 2024-08-15T16:59:35.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 -- start start 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2104153320 0x7f21041536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f210415a190 0x7f210415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210415dbd0 con 0x7f21040ccaa0 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21040ce430 con 0x7f210415a190 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210b4a1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210415d370 con 0x7f2104153320 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210aca0700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f21040ccaa0 0x7f2104066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:56164/0 (socket says 172.21.15.146:56164) 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210aca0700 1 -- 172.21.15.146:0/1092994380 learned_addr learned my addr 172.21.15.146:0/1092994380 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f210a49f700 1 --2- 172.21.15.146:0/1092994380 >> v2:172.21.15.161:3300/0 conn(0x7f2104153320 0x7f21041536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.853+0000 7f2109c9e700 1 --2- 172.21.15.146:0/1092994380 >> v2:172.21.15.146:3300/0 conn(0x7f210415a190 0x7f210415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210949d700 1 -- 172.21.15.146:0/1092994380 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814200196 0 0) 0x7f210415dbd0 con 0x7f21040ccaa0 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210949d700 1 -- 172.21.15.146:0/1092994380 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f0003570 con 0x7f21040ccaa0 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210a49f700 1 -- 172.21.15.146:0/1092994380 >> v2:172.21.15.146:3300/0 conn(0x7f210415a190 msgr2=0x7f210415cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210a49f700 1 --2- 172.21.15.146:0/1092994380 >> v2:172.21.15.146:3300/0 conn(0x7f210415a190 0x7f210415cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210a49f700 1 -- 172.21.15.146:0/1092994380 >> v1:172.21.15.146:6789/0 conn(0x7f21040ccaa0 legacy=0x7f2104066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210a49f700 1 -- 172.21.15.146:0/1092994380 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210415a590 con 0x7f2104153320 2024-08-15T16:59:35.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210a49f700 1 --2- 172.21.15.146:0/1092994380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2104153320 0x7f21041536f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21040d0cd0 tx=0x7f21000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d691577e65d4e91 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210949d700 1 -- 172.21.15.146:0/1092994380 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100011630 con 0x7f2104153320 2024-08-15T16:59:35.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210949d700 1 -- 172.21.15.146:0/1092994380 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21000117b0 con 0x7f2104153320 2024-08-15T16:59:35.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210949d700 1 -- 172.21.15.146:0/1092994380 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100011b70 con 0x7f2104153320 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- 172.21.15.146:0/1092994380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2104153320 msgr2=0x7f21041536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 --2- 172.21.15.146:0/1092994380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2104153320 0x7f21041536f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f21040d0cd0 tx=0x7f21000091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- 172.21.15.146:0/1092994380 shutdown_connections 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 --2- 172.21.15.146:0/1092994380 >> v2:172.21.15.146:3300/0 conn(0x7f210415a190 0x7f210415cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 --2- 172.21.15.146:0/1092994380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2104153320 0x7f21041536f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- 172.21.15.146:0/1092994380 >> 172.21.15.146:0/1092994380 conn(0x7f21040d00a0 msgr2=0x7f21040d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- 172.21.15.146:0/1092994380 shutdown_connections 2024-08-15T16:59:35.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- 172.21.15.146:0/1092994380 wait complete. 2024-08-15T16:59:35.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 Processor -- start 2024-08-15T16:59:35.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.857+0000 7f210b4a1700 1 -- start start 2024-08-15T16:59:35.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 0x7f2104169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 0x7f2104169b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210416a0d0 con 0x7f21040ccaa0 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21040c5b60 con 0x7f210415a190 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21040c4f50 con 0x7f2104153320 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60988/0 (socket says 172.21.15.146:60988) 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 -- 172.21.15.146:0/265490502 learned_addr learned my addr 172.21.15.146:0/265490502 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210aca0700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 0x7f2104169b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f2109c9e700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 0x7f2104169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 -- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 msgr2=0x7f2104169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 0x7f2104169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 -- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 msgr2=0x7f2104169b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 0x7f2104169b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 -- 172.21.15.146:0/265490502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100009420 con 0x7f21040ccaa0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210aca0700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 0x7f2104169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f2109c9e700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 0x7f2104169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210a49f700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2100013000 tx=0x7f2100017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210001f070 con 0x7f21040ccaa0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2100000bc0 con 0x7f21040ccaa0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21040c5230 con 0x7f21040ccaa0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100011b70 con 0x7f21040ccaa0 2024-08-15T16:59:35.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21040c57f0 con 0x7f21040ccaa0 2024-08-15T16:59:35.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21040638f0 con 0x7f21040ccaa0 2024-08-15T16:59:35.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.861+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f2100023070 con 0x7f21040ccaa0 2024-08-15T16:59:35.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.865+0000 7f20fa7fc700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 0x7f20e8062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:35.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.865+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f210007d920 con 0x7f21040ccaa0 2024-08-15T16:59:35.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.865+0000 7f2109c9e700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 0x7f20e8062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:35.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.865+0000 7f2109c9e700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 0x7f20e8062c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21040c6b10 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:35.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:35.869+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f210004e320 con 0x7f21040ccaa0 2024-08-15T16:59:36.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.033+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21040cde20 con 0x7f21040ccaa0 2024-08-15T16:59:36.048 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f20fa7fc700 1 -- 172.21.15.146:0/265490502 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10444 (secure 0 0 0) 0x7f2100015090 con 0x7f21040ccaa0 2024-08-15T16:59:36.048 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:36.048 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":14,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:35.356214+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:36.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 msgr2=0x7f20e8062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 0x7f20e8062c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21040c6b10 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2024-08-15T16:59:36.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 msgr2=0x7f21040c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2100013000 tx=0x7f2100017000 comp rx=0 tx=0).stop 2024-08-15T16:59:36.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 shutdown_connections 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f20e80607d0 0x7f20e8062c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f210415a190 0x7f2104169b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2104153320 0x7f2104169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 --2- 172.21.15.146:0/265490502 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f21040ccaa0 0x7f21040c5ff0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 >> 172.21.15.146:0/265490502 conn(0x7f21040d00a0 msgr2=0x7f210415ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 shutdown_connections 2024-08-15T16:59:36.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.037+0000 7f210b4a1700 1 -- 172.21.15.146:0/265490502 wait complete. 2024-08-15T16:59:36.063 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-08-15T16:59:36.063 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:36.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 Processor -- start 2024-08-15T16:59:36.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 -- start start 2024-08-15T16:59:36.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6964155d20 0x7f69640b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 0x7f69640aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f696406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69640adc10 con 0x7f6964155d20 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f696406e8b0 con 0x7f69640ad7c0 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969f4a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69640af2b0 con 0x7f696406df10 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6968f48700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6964155d20 0x7f69640b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6968f48700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6964155d20 0x7f69640b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60992/0 (socket says 172.21.15.146:60992) 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6968f48700 1 -- 172.21.15.146:0/3038066024 learned_addr learned my addr 172.21.15.146:0/3038066024 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6969749700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f696406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.157+0000 7f6963fff700 1 --2- 172.21.15.146:0/3038066024 >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 0x7f69640aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 -- 172.21.15.146:0/3038066024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 msgr2=0x7f696406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:36.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f696406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 -- 172.21.15.146:0/3038066024 >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 msgr2=0x7f69640aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:36.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 --2- 172.21.15.146:0/3038066024 >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 0x7f69640aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 -- 172.21.15.146:0/3038066024 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69640af590 con 0x7f6964155d20 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6963fff700 1 --2- 172.21.15.146:0/3038066024 >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 0x7f69640aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6968f48700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6964155d20 0x7f69640b0350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f696406fb30 tx=0x7f6958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af4df2f58ff9899 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f69637fe700 1 -- 172.21.15.146:0/3038066024 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958009e70 con 0x7f6964155d20 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f69637fe700 1 -- 172.21.15.146:0/3038066024 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695800b860 con 0x7f6964155d20 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f69637fe700 1 -- 172.21.15.146:0/3038066024 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695800bc00 con 0x7f6964155d20 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 -- 172.21.15.146:0/3038066024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6964155d20 msgr2=0x7f69640b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6964155d20 0x7f69640b0350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f696406fb30 tx=0x7f6958009250 comp rx=0 tx=0).stop 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 -- 172.21.15.146:0/3038066024 shutdown_connections 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f696406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3038066024 >> v2:172.21.15.161:3300/0 conn(0x7f69640ad7c0 0x7f69640aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3038066024 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6964155d20 0x7f69640b0350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 -- 172.21.15.146:0/3038066024 >> 172.21.15.146:0/3038066024 conn(0x7f69640678d0 msgr2=0x7f6964067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 -- 172.21.15.146:0/3038066024 shutdown_connections 2024-08-15T16:59:36.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.161+0000 7f6969f4a700 1 -- 172.21.15.146:0/3038066024 wait complete. 2024-08-15T16:59:36.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 Processor -- start 2024-08-15T16:59:36.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- start start 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f6964146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 0x7f69641696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964142c50 con 0x7f69640ad7c0 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964141fd0 con 0x7f6964155d20 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69641422b0 con 0x7f696406df10 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:60994/0 (socket says 172.21.15.146:60994) 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969749700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 0x7f69641696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 -- 172.21.15.146:0/3441653944 learned_addr learned my addr 172.21.15.146:0/3441653944 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:36.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6968f48700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f6964146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 -- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 msgr2=0x7f6964146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f6964146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 -- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 msgr2=0x7f69641696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 0x7f69641696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 -- 172.21.15.146:0/3441653944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580094f0 con 0x7f69640ad7c0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969749700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 0x7f69641696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6963fff700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f695400e8d0 tx=0x7f6954008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954013410 con 0x7f69640ad7c0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6954013570 con 0x7f69640ad7c0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964142590 con 0x7f69640ad7c0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6954013910 con 0x7f69640ad7c0 2024-08-15T16:59:36.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.165+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6964147060 con 0x7f69640ad7c0 2024-08-15T16:59:36.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f6954013d40 con 0x7f69640ad7c0 2024-08-15T16:59:36.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69640643a0 con 0x7f69640ad7c0 2024-08-15T16:59:36.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6960ff9700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 0x7f6950062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:36.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f695407c310 con 0x7f69640ad7c0 2024-08-15T16:59:36.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6968f48700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 0x7f6950062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:36.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.169+0000 7f6968f48700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 0x7f6950062c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f696406fb30 tx=0x7f6958017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:36.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.177+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f695404cc70 con 0x7f69640ad7c0 2024-08-15T16:59:36.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.377+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f696406f5f0 con 0x7f69640ad7c0 2024-08-15T16:59:36.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6960ff9700 1 -- 172.21.15.146:0/3441653944 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7f6954020500 con 0x7f69640ad7c0 2024-08-15T16:59:36.392 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:36.392 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 msgr2=0x7f6950062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 0x7f6950062c80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f696406fb30 tx=0x7f6958017000 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 msgr2=0x7f6964148250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f695400e8d0 tx=0x7f6954008040 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 shutdown_connections 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69500607d0 0x7f6950062c80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6964155d20 0x7f69641696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69640ad7c0 0x7f6964148250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 --2- 172.21.15.146:0/3441653944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f696406df10 0x7f6964146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 >> 172.21.15.146:0/3441653944 conn(0x7f69640678d0 msgr2=0x7f696414bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:36.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 shutdown_connections 2024-08-15T16:59:36.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:36.381+0000 7f6969f4a700 1 -- 172.21.15.146:0/3441653944 wait complete. 2024-08-15T16:59:36.396 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 2 2024-08-15T16:59:38.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:38.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.501+0000 7f35c2237700 1 Processor -- start 2024-08-15T16:59:38.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.501+0000 7f35c2237700 1 -- start start 2024-08-15T16:59:38.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc156c40 0x7f35bc0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35bc068db0 0x7f35bc0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc1585b0 con 0x7f35bc157010 2024-08-15T16:59:38.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0c9fd0 con 0x7f35bc068db0 2024-08-15T16:59:38.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc1579c0 con 0x7f35bc156c40 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35bc068db0 0x7f35bc0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f35bc068db0 0x7f35bc0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50572/0 (socket says 172.21.15.146:50572) 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 -- 172.21.15.146:0/93453583 learned_addr learned my addr 172.21.15.146:0/93453583 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1235700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc156c40 0x7f35bc0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 -- 172.21.15.146:0/93453583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc156c40 msgr2=0x7f35bc0cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc156c40 0x7f35bc0cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 -- 172.21.15.146:0/93453583 >> v1:172.21.15.161:6789/0 conn(0x7f35bc157010 legacy=0x7f35bc0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 -- 172.21.15.146:0/93453583 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc157c80 con 0x7f35bc068db0 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c1a36700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc068db0 0x7f35bc0ca1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009f50 tx=0x7f35b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9835f33d9e5291 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35b3fff700 1 -- 172.21.15.146:0/93453583 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b400e070 con 0x7f35bc068db0 2024-08-15T16:59:38.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35b3fff700 1 -- 172.21.15.146:0/93453583 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b400b9c0 con 0x7f35bc068db0 2024-08-15T16:59:38.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35b3fff700 1 -- 172.21.15.146:0/93453583 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40092e0 con 0x7f35bc068db0 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- 172.21.15.146:0/93453583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc068db0 msgr2=0x7f35bc0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc068db0 0x7f35bc0ca1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009f50 tx=0x7f35b4009840 comp rx=0 tx=0).stop 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- 172.21.15.146:0/93453583 shutdown_connections 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc068db0 0x7f35bc0ca1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 --2- 172.21.15.146:0/93453583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc156c40 0x7f35bc0cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- 172.21.15.146:0/93453583 >> 172.21.15.146:0/93453583 conn(0x7f35bc06d800 msgr2=0x7f35bc06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:38.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- 172.21.15.146:0/93453583 shutdown_connections 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- 172.21.15.146:0/93453583 wait complete. 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 Processor -- start 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.505+0000 7f35c2237700 1 -- start start 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 0x7f35bc14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 0x7f35bc0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc14db00 con 0x7f35bc157010 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0b14d0 con 0x7f35bc156c40 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0c5f70 con 0x7f35bc068db0 2024-08-15T16:59:38.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50584/0 (socket says 172.21.15.146:50584) 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 -- 172.21.15.146:0/4160513268 learned_addr learned my addr 172.21.15.146:0/4160513268 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1235700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 0x7f35bc14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c0a34700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 0x7f35bc0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 -- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 msgr2=0x7f35bc14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 0x7f35bc14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 -- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 msgr2=0x7f35bc0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 0x7f35bc0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 -- 172.21.15.146:0/4160513268 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b4009ae0 con 0x7f35bc157010 2024-08-15T16:59:38.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1235700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 0x7f35bc14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c0a34700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 0x7f35bc0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c1a36700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009880 tx=0x7f35b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4008d00 con 0x7f35bc157010 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc14d820 con 0x7f35bc157010 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4008e60 con 0x7f35bc157010 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4026690 con 0x7f35bc157010 2024-08-15T16:59:38.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc0c5c00 con 0x7f35bc157010 2024-08-15T16:59:38.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc0c5780 con 0x7f35bc157010 2024-08-15T16:59:38.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.509+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f35b4003040 con 0x7f35bc157010 2024-08-15T16:59:38.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.513+0000 7f35b17fa700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 0x7f35a0062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.513+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f35b407ebc0 con 0x7f35bc157010 2024-08-15T16:59:38.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.517+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f35bc0c5780 con 0x7f35bc157010 2024-08-15T16:59:38.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.533+0000 7f35c1235700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 0x7f35a0062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.533+0000 7f35c1235700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 0x7f35a0062c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009d80 tx=0x7f35b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.721+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35bc0c5780 con 0x7f35bc157010 2024-08-15T16:59:38.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.721+0000 7f35b17fa700 1 -- 172.21.15.146:0/4160513268 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f35b402bde0 con 0x7f35bc157010 2024-08-15T16:59:38.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:38.734 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4245,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 msgr2=0x7f35a0062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 0x7f35a0062c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009d80 tx=0x7f35b8008040 comp rx=0 tx=0).stop 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 msgr2=0x7f35bc14a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009880 tx=0x7f35b4018000 comp rx=0 tx=0).stop 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 shutdown_connections 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f35a00607d0 0x7f35a0062c80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f35bc157010 0x7f35bc14a980 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f35bc156c40 0x7f35bc0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 --2- 172.21.15.146:0/4160513268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f35bc068db0 0x7f35bc14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 >> 172.21.15.146:0/4160513268 conn(0x7f35bc06d800 msgr2=0x7f35bc0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 shutdown_connections 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.725+0000 7f35c2237700 1 -- 172.21.15.146:0/4160513268 wait complete. 2024-08-15T16:59:38.738 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 3 2024-08-15T16:59:38.749 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:38.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.829+0000 7f4ea359e700 1 Processor -- start 2024-08-15T16:59:38.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.829+0000 7f4ea359e700 1 -- start start 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ea4142b50 0x7f4ea40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40ca510 con 0x7f4ea4142780 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40cb290 con 0x7f4ea40cae50 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40c9900 con 0x7f4ea4142b50 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ea4142b50 0x7f4ea40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f4ea4142b50 0x7f4ea40ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50592/0 (socket says 172.21.15.146:50592) 2024-08-15T16:59:38.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1491454785 learned_addr learned my addr 172.21.15.146:0/1491454785 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:38.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1491454785 >> v1:172.21.15.161:6789/0 conn(0x7f4ea40cae50 legacy=0x7f4ea415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1491454785 >> v1:172.21.15.146:6789/0 conn(0x7f4ea4142780 legacy=0x7f4ea40ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1491454785 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40c9be0 con 0x7f4ea4142b50 2024-08-15T16:59:38.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea1d9b700 1 --2- 172.21.15.146:0/1491454785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40ca910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e980099b0 tx=0x7f4e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c11fe61525fcb21 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea159a700 1 -- 172.21.15.146:0/1491454785 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800b860 con 0x7f4ea4142b50 2024-08-15T16:59:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea159a700 1 -- 172.21.15.146:0/1491454785 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800b9c0 con 0x7f4ea4142b50 2024-08-15T16:59:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea159a700 1 -- 172.21.15.146:0/1491454785 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800bd60 con 0x7f4ea4142b50 2024-08-15T16:59:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- 172.21.15.146:0/1491454785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 msgr2=0x7f4ea40ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1491454785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40ca910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e980099b0 tx=0x7f4e98009250 comp rx=0 tx=0).stop 2024-08-15T16:59:38.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- 172.21.15.146:0/1491454785 shutdown_connections 2024-08-15T16:59:38.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1491454785 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40ca910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- 172.21.15.146:0/1491454785 >> 172.21.15.146:0/1491454785 conn(0x7f4ea406e070 msgr2=0x7f4ea40b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:38.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- 172.21.15.146:0/1491454785 shutdown_connections 2024-08-15T16:59:38.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.833+0000 7f4ea359e700 1 -- 172.21.15.146:0/1491454785 wait complete. 2024-08-15T16:59:38.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 Processor -- start 2024-08-15T16:59:38.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 -- start start 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ea40cae50 0x7f4ea40a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea413f2f0 con 0x7f4ea4142b50 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40c42a0 con 0x7f4ea4142780 2024-08-15T16:59:38.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.837+0000 7f4ea359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40c4580 con 0x7f4ea40cae50 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea2d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ea40cae50 0x7f4ea40a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:60132/0 (socket says 172.21.15.146:60132) 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1676700484 learned_addr learned my addr 172.21.15.146:0/1676700484 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ea40cae50 msgr2=0x7f4ea40a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ea40cae50 0x7f4ea40a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1676700484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 msgr2=0x7f4ea40c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 -- 172.21.15.146:0/1676700484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e980094f0 con 0x7f4ea4142780 2024-08-15T16:59:38.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea2d9d700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea1d9b700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ea406fbe0 tx=0x7f4e98017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98024440 con 0x7f4ea4142780 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e980245a0 con 0x7f4ea4142780 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea40c4860 con 0x7f4ea4142780 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98024940 con 0x7f4ea4142780 2024-08-15T16:59:38.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40c4e20 con 0x7f4ea4142780 2024-08-15T16:59:38.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea40c5010 con 0x7f4ea4142780 2024-08-15T16:59:38.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.841+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f4e9801a420 con 0x7f4ea4142780 2024-08-15T16:59:38.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.845+0000 7f4e927fc700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 0x7f4e84062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:38.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.845+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4e9807e570 con 0x7f4ea4142780 2024-08-15T16:59:38.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.845+0000 7f4ea259c700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 0x7f4e84062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:38.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.845+0000 7f4ea259c700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 0x7f4e84062c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e94003a80 tx=0x7f4e94007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:38.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:38.857+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4e9804ee00 con 0x7f4ea4142780 2024-08-15T16:59:39.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.021+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ea40638f0 con 0x7f4ea4142780 2024-08-15T16:59:39.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.025+0000 7f4e927fc700 1 -- 172.21.15.146:0/1676700484 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f4e98052420 con 0x7f4ea4142780 2024-08-15T16:59:39.036 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:39.037 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":17,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:38.364868+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:39.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 msgr2=0x7f4e84062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 0x7f4e84062c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4e94003a80 tx=0x7f4e94007ec0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 msgr2=0x7f4ea40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ea406fbe0 tx=0x7f4e98017000 comp rx=0 tx=0).stop 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 shutdown_connections 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4e840607d0 0x7f4e84062c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ea4142b50 0x7f4ea40c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ea4142780 0x7f4ea40a5990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 --2- 172.21.15.146:0/1676700484 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ea40cae50 0x7f4ea40a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 >> 172.21.15.146:0/1676700484 conn(0x7f4ea406e070 msgr2=0x7f4ea415b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 shutdown_connections 2024-08-15T16:59:39.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.029+0000 7f4ea359e700 1 -- 172.21.15.146:0/1676700484 wait complete. 2024-08-15T16:59:39.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T16:59:39.054 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T16:59:39.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.153+0000 7f1c7759e700 1 Processor -- start 2024-08-15T16:59:39.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.153+0000 7f1c7759e700 1 -- start start 2024-08-15T16:59:39.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7813be40 con 0x7f1c7806a040 2024-08-15T16:59:39.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7813d930 con 0x7f1c78155c30 2024-08-15T16:59:39.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7813b230 con 0x7f1c78156000 2024-08-15T16:59:39.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c76d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1c7806a040 0x7f1c7813aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34984/0 (socket says 172.21.15.146:34984) 2024-08-15T16:59:39.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c76d9d700 1 -- 172.21.15.146:0/3336203513 learned_addr learned my addr 172.21.15.146:0/3336203513 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:39.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c6ffff700 1 --2- 172.21.15.146:0/3336203513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c75d9b700 1 -- 172.21.15.146:0/3336203513 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421365244 0 0) 0x7f1c7813d930 con 0x7f1c78155c30 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c75d9b700 1 -- 172.21.15.146:0/3336203513 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c64003570 con 0x7f1c78155c30 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c6ffff700 1 -- 172.21.15.146:0/3336203513 >> v1:172.21.15.161:6789/0 conn(0x7f1c78155c30 legacy=0x7f1c78069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c6ffff700 1 -- 172.21.15.146:0/3336203513 >> v1:172.21.15.146:6789/0 conn(0x7f1c7806a040 legacy=0x7f1c7813aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c6ffff700 1 -- 172.21.15.146:0/3336203513 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7813b510 con 0x7f1c78156000 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c6ffff700 1 --2- 172.21.15.146:0/3336203513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a4c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009910 tx=0x7f1c600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de814f357cecbab0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c75d9b700 1 -- 172.21.15.146:0/3336203513 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000f6b0 con 0x7f1c78156000 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c75d9b700 1 -- 172.21.15.146:0/3336203513 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6000f810 con 0x7f1c78156000 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c75d9b700 1 -- 172.21.15.146:0/3336203513 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000fb90 con 0x7f1c78156000 2024-08-15T16:59:39.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- 172.21.15.146:0/3336203513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 msgr2=0x7f1c7813a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 --2- 172.21.15.146:0/3336203513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a4c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009910 tx=0x7f1c600091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- 172.21.15.146:0/3336203513 shutdown_connections 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 --2- 172.21.15.146:0/3336203513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a4c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- 172.21.15.146:0/3336203513 >> 172.21.15.146:0/3336203513 conn(0x7f1c7806e170 msgr2=0x7f1c7806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- 172.21.15.146:0/3336203513 shutdown_connections 2024-08-15T16:59:39.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.157+0000 7f1c7759e700 1 -- 172.21.15.146:0/3336203513 wait complete. 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 Processor -- start 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- start start 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 0x7f1c78154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 0x7f1c780c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c78157b30 con 0x7f1c78155c30 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c78156f60 con 0x7f1c7806a040 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c78157200 con 0x7f1c78156000 2024-08-15T16:59:39.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 0x7f1c780c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 0x7f1c780c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50612/0 (socket says 172.21.15.146:50612) 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6ffff700 1 -- 172.21.15.146:0/2751576944 learned_addr learned my addr 172.21.15.146:0/2751576944 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c76d9d700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 0x7f1c78154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 -- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 msgr2=0x7f1c7813a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 -- 172.21.15.146:0/2751576944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 msgr2=0x7f1c780c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 0x7f1c780c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 -- 172.21.15.146:0/2751576944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c60009420 con 0x7f1c7806a040 2024-08-15T16:59:39.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7659c700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 0x7f1c78154260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c68007d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011590 con 0x7f1c7806a040 2024-08-15T16:59:39.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680116f0 con 0x7f1c7806a040 2024-08-15T16:59:39.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c781574e0 con 0x7f1c7806a040 2024-08-15T16:59:39.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011ab0 con 0x7f1c7806a040 2024-08-15T16:59:39.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.161+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c780cc6c0 con 0x7f1c7806a040 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c780638f0 con 0x7f1c7806a040 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f1c6801c460 con 0x7f1c7806a040 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c6e7fc700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 0x7f1c58062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1c6807c2a0 con 0x7f1c7806a040 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c6ffff700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 0x7f1c58062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.165+0000 7f1c6ffff700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 0x7f1c58062cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000ffd0 tx=0x7f1c60019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.173+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f1c6804cb20 con 0x7f1c7806a040 2024-08-15T16:59:39.365 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.353+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c780708e0 con 0x7f1c7806a040 2024-08-15T16:59:39.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.353+0000 7f1c6e7fc700 1 -- 172.21.15.146:0/2751576944 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c68050140 con 0x7f1c7806a040 2024-08-15T16:59:39.366 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 msgr2=0x7f1c58062cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 0x7f1c58062cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000ffd0 tx=0x7f1c60019000 comp rx=0 tx=0).stop 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 msgr2=0x7f1c78154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 0x7f1c78154260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c68007d30 comp rx=0 tx=0).stop 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 shutdown_connections 2024-08-15T16:59:39.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1c58060800 0x7f1c58062cb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1c78156000 0x7f1c7813a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c78155c30 0x7f1c780c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 --2- 172.21.15.146:0/2751576944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1c7806a040 0x7f1c78154260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 >> 172.21.15.146:0/2751576944 conn(0x7f1c7806e170 msgr2=0x7f1c78151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 shutdown_connections 2024-08-15T16:59:39.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.357+0000 7f1c7759e700 1 -- 172.21.15.146:0/2751576944 wait complete. 2024-08-15T16:59:39.384 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T16:59:39.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.477+0000 7f7dbad3c700 1 Processor -- start 2024-08-15T16:59:39.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.477+0000 7f7dbad3c700 1 -- start start 2024-08-15T16:59:39.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 0x7f7db4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7db415a090 0x7f7db40ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db40c9e00 con 0x7f7db4153a40 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40ccd20 con 0x7f7db415a090 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40cc890 con 0x7f7db40c9820 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7db415a090 0x7f7db40ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f7db415a090 0x7f7db40ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50624/0 (socket says 172.21.15.146:50624) 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 -- 172.21.15.146:0/1832614306 learned_addr learned my addr 172.21.15.146:0/1832614306 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dabfff700 1 --2- 172.21.15.146:0/1832614306 >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 0x7f7db4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 -- 172.21.15.146:0/1832614306 >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 msgr2=0x7f7db4159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 --2- 172.21.15.146:0/1832614306 >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 0x7f7db4159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 -- 172.21.15.146:0/1832614306 >> v1:172.21.15.161:6789/0 conn(0x7f7db4153a40 legacy=0x7f7db40b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 -- 172.21.15.146:0/1832614306 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db415a480 con 0x7f7db415a090 2024-08-15T16:59:39.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dabfff700 1 --2- 172.21.15.146:0/1832614306 >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 0x7f7db4159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:39.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7db92d9700 1 --2- 172.21.15.146:0/1832614306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db40ada90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ccf10 tx=0x7f7dac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f5d99ff571918d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dab7fe700 1 -- 172.21.15.146:0/1832614306 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac010070 con 0x7f7db415a090 2024-08-15T16:59:39.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dab7fe700 1 -- 172.21.15.146:0/1832614306 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00de90 con 0x7f7db415a090 2024-08-15T16:59:39.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dab7fe700 1 -- 172.21.15.146:0/1832614306 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00b320 con 0x7f7db415a090 2024-08-15T16:59:39.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- 172.21.15.146:0/1832614306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 msgr2=0x7f7db40ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/1832614306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db40ada90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7db40ccf10 tx=0x7f7dac00ba50 comp rx=0 tx=0).stop 2024-08-15T16:59:39.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- 172.21.15.146:0/1832614306 shutdown_connections 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/1832614306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db40ada90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/1832614306 >> v2:172.21.15.161:3300/0 conn(0x7f7db40c9820 0x7f7db4159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- 172.21.15.146:0/1832614306 >> 172.21.15.146:0/1832614306 conn(0x7f7db40a9db0 msgr2=0x7f7db40ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- 172.21.15.146:0/1832614306 shutdown_connections 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.481+0000 7f7dbad3c700 1 -- 172.21.15.146:0/1832614306 wait complete. 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 Processor -- start 2024-08-15T16:59:39.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 -- start start 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7db40c9820 0x7f7db414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7db4153a40 0x7f7db4098fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41510b0 con 0x7f7db415a090 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4099b70 con 0x7f7db40c9820 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dbad3c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4150780 con 0x7f7db4153a40 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7db92d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7db92d9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50628/0 (socket says 172.21.15.146:50628) 2024-08-15T16:59:39.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7db92d9700 1 -- 172.21.15.146:0/3507155564 learned_addr learned my addr 172.21.15.146:0/3507155564 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7db8ad8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7db40c9820 0x7f7db414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.485+0000 7f7dabfff700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7db4153a40 0x7f7db4098fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 -- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7db4153a40 msgr2=0x7f7db4098fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7db4153a40 0x7f7db4098fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 -- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7db40c9820 msgr2=0x7f7db414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7db40c9820 0x7f7db414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 -- 172.21.15.146:0/3507155564 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac00bcf0 con 0x7f7db415a090 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7db92d9700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7db40704b0 tx=0x7f7dac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00ac00 con 0x7f7db415a090 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00ad80 con 0x7f7db415a090 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4150a60 con 0x7f7db415a090 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac01e690 con 0x7f7db415a090 2024-08-15T16:59:39.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db414fb70 con 0x7f7db415a090 2024-08-15T16:59:39.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db4159550 con 0x7f7db415a090 2024-08-15T16:59:39.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7f7dac003040 con 0x7f7db415a090 2024-08-15T16:59:39.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.489+0000 7f7da8ff9700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 0x7f7da0062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.493+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7dac07e240 con 0x7f7db415a090 2024-08-15T16:59:39.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.493+0000 7f7db8ad8700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 0x7f7da0062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.493+0000 7f7db8ad8700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 0x7f7da0062c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7db0003950 tx=0x7f7db0007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.497+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7db4159550 con 0x7f7db415a090 2024-08-15T16:59:39.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.661+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7db4159550 con 0x7f7db415a090 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.661+0000 7f7da8ff9700 1 -- 172.21.15.146:0/3507155564 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7dac008060 con 0x7f7db415a090 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout:[ 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout: "poolnum": 1, 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout: "poolname": ".mgr" 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T16:59:39.674 INFO:teuthology.orchestra.run.smithi146.stdout:] 2024-08-15T16:59:39.676 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 msgr2=0x7f7da0062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 0x7f7da0062c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7db0003950 tx=0x7f7db0007ec0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 msgr2=0x7f7db4099510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7db40704b0 tx=0x7f7dac01a000 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 shutdown_connections 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7da00607d0 0x7f7da0062c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7db415a090 0x7f7db4099510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7db4153a40 0x7f7db4098fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 --2- 172.21.15.146:0/3507155564 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7db40c9820 0x7f7db414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 >> 172.21.15.146:0/3507155564 conn(0x7f7db40a9db0 msgr2=0x7f7db4158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 shutdown_connections 2024-08-15T16:59:39.677 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.665+0000 7f7dbad3c700 1 -- 172.21.15.146:0/3507155564 wait complete. 2024-08-15T16:59:39.688 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T16:59:39.688 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T16:59:39.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.793+0000 7fde17764700 1 Processor -- start 2024-08-15T16:59:39.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.793+0000 7fde17764700 1 -- start start 2024-08-15T16:59:39.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde100cd2b0 0x7fde100cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100cc480 con 0x7fde100caeb0 2024-08-15T16:59:39.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde10159720 con 0x7fde100cd2b0 2024-08-15T16:59:39.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100cb870 con 0x7fde100ceb70 2024-08-15T16:59:39.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde100cd2b0 0x7fde100cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15d01700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde14cff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fde100caeb0 0x7fde100ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:43584/0 (socket says 172.21.15.146:43584) 2024-08-15T16:59:39.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15d01700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:56636/0 (socket says 172.21.15.146:56636) 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde14cff700 1 -- 172.21.15.146:0/2829605406 learned_addr learned my addr 172.21.15.146:0/2829605406 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde100cd2b0 0x7fde100cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:60188/0 (socket says 172.21.15.146:60188) 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde07fff700 1 -- 172.21.15.146:0/2829605406 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371218038 0 0) 0x7fde100cc480 con 0x7fde100caeb0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde07fff700 1 -- 172.21.15.146:0/2829605406 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddfc003570 con 0x7fde100caeb0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 -- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 msgr2=0x7fde10162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 -- 172.21.15.146:0/2829605406 >> v1:172.21.15.161:6789/0 conn(0x7fde100caeb0 legacy=0x7fde100ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 -- 172.21.15.146:0/2829605406 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100cbb50 con 0x7fde100cd2b0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15d01700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde15500700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100cd2b0 0x7fde100cd680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde000098e0 tx=0x7fde000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aef1e07257eef23 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde07fff700 1 -- 172.21.15.146:0/2829605406 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00011630 con 0x7fde100cd2b0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde07fff700 1 -- 172.21.15.146:0/2829605406 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00011790 con 0x7fde100cd2b0 2024-08-15T16:59:39.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde07fff700 1 -- 172.21.15.146:0/2829605406 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00011b30 con 0x7fde100cd2b0 2024-08-15T16:59:39.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100cd2b0 msgr2=0x7fde100cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100cd2b0 0x7fde100cd680 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde000098e0 tx=0x7fde000091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- 172.21.15.146:0/2829605406 shutdown_connections 2024-08-15T16:59:39.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde10162ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 --2- 172.21.15.146:0/2829605406 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100cd2b0 0x7fde100cd680 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.797+0000 7fde17764700 1 -- 172.21.15.146:0/2829605406 >> 172.21.15.146:0/2829605406 conn(0x7fde10099550 msgr2=0x7fde10067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:39.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.801+0000 7fde17764700 1 -- 172.21.15.146:0/2829605406 shutdown_connections 2024-08-15T16:59:39.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.801+0000 7fde17764700 1 -- 172.21.15.146:0/2829605406 wait complete. 2024-08-15T16:59:39.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.801+0000 7fde17764700 1 Processor -- start 2024-08-15T16:59:39.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.801+0000 7fde17764700 1 -- start start 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100caeb0 0x7fde100b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde100c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde101570c0 con 0x7fde100cd2b0 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100c84e0 con 0x7fde100caeb0 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100b42f0 con 0x7fde100ceb70 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde14cff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde14cff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50636/0 (socket says 172.21.15.146:50636) 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde14cff700 1 -- 172.21.15.146:0/2043987550 learned_addr learned my addr 172.21.15.146:0/2043987550 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15500700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100caeb0 0x7fde100b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde100c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 -- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100caeb0 msgr2=0x7fde100b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100caeb0 0x7fde100b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 -- 172.21.15.146:0/2043987550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 msgr2=0x7fde10156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 -- 172.21.15.146:0/2043987550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08009a20 con 0x7fde100ceb70 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde14cff700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde15d01700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde100c7fa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde10099c80 tx=0x7fde0800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800ca40 con 0x7fde100ceb70 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800f040 con 0x7fde100ceb70 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800ce10 con 0x7fde100ceb70 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde00009420 con 0x7fde100ceb70 2024-08-15T16:59:39.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100b39c0 con 0x7fde100ceb70 2024-08-15T16:59:39.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.805+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde100b4480 con 0x7fde100ceb70 2024-08-15T16:59:39.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.809+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 4) v1 ==== 80087+0+0 (secure 0 0 0) 0x7fde08014690 con 0x7fde100ceb70 2024-08-15T16:59:39.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.809+0000 7fde057fa700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 0x7fddf4062c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:39.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.809+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fde0807d850 con 0x7fde100ceb70 2024-08-15T16:59:39.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.809+0000 7fde15500700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 0x7fddf4062c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:39.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.809+0000 7fde15500700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 0x7fddf4062c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde000091e0 tx=0x7fde00017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:39.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.821+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fde0804e150 con 0x7fde100ceb70 2024-08-15T16:59:39.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:39.981+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fde10068020 con 0x7fde100ceb70 2024-08-15T16:59:40.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.365+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fde0804e520 con 0x7fde100ceb70 2024-08-15T16:59:40.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.365+0000 7fde057fa700 1 -- 172.21.15.146:0/2043987550 <== mon.2 v2:172.21.15.161:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fde080283b0 con 0x7fde100ceb70 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.369+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 msgr2=0x7fddf4062c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.369+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 0x7fddf4062c80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fde000091e0 tx=0x7fde00017000 comp rx=0 tx=0).stop 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.369+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 msgr2=0x7fde100c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.369+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde100c7fa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde10099c80 tx=0x7fde0800b040 comp rx=0 tx=0).stop 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 shutdown_connections 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fddf40607d0 0x7fddf4062c80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde100ceb70 0x7fde100c7fa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fde100cd2b0 0x7fde10156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 --2- 172.21.15.146:0/2043987550 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde100caeb0 0x7fde100b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 >> 172.21.15.146:0/2043987550 conn(0x7fde10099550 msgr2=0x7fde101588b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:40.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 shutdown_connections 2024-08-15T16:59:40.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.373+0000 7fde17764700 1 -- 172.21.15.146:0/2043987550 wait complete. 2024-08-15T16:59:40.385 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' created 2024-08-15T16:59:40.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T16:59:40.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.497+0000 7fa2d3168700 1 Processor -- start 2024-08-15T16:59:40.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.497+0000 7fa2d3168700 1 -- start start 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2cc0adec0 0x7fa2cc0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa2cc06e8c0 0x7fa2cc13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0aef90 con 0x7fa2cc0aeb10 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2cc1573c0 con 0x7fa2cc0adec0 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2cc156480 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2cc0adec0 0x7fa2cc0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa2cc0adec0 0x7fa2cc0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50640/0 (socket says 172.21.15.146:50640) 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 -- 172.21.15.146:0/4175838982 learned_addr learned my addr 172.21.15.146:0/4175838982 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:40.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2967700 1 --2- 172.21.15.146:0/4175838982 >> v2:172.21.15.161:3300/0 conn(0x7fa2cc06e8c0 0x7fa2cc13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 -- 172.21.15.146:0/4175838982 >> v2:172.21.15.161:3300/0 conn(0x7fa2cc06e8c0 msgr2=0x7fa2cc13f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 --2- 172.21.15.146:0/4175838982 >> v2:172.21.15.161:3300/0 conn(0x7fa2cc06e8c0 0x7fa2cc13f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 -- 172.21.15.146:0/4175838982 >> v1:172.21.15.161:6789/0 conn(0x7fa2cc0aeb10 legacy=0x7fa2cc06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 -- 172.21.15.146:0/4175838982 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc156740 con 0x7fa2cc0adec0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d2166700 1 --2- 172.21.15.146:0/4175838982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc0ae290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009f20 tx=0x7fa2c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78eb422157147e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d1164700 1 -- 172.21.15.146:0/4175838982 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400e070 con 0x7fa2cc0adec0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d1164700 1 -- 172.21.15.146:0/4175838982 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c400b990 con 0x7fa2cc0adec0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d1164700 1 -- 172.21.15.146:0/4175838982 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40092e0 con 0x7fa2cc0adec0 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- 172.21.15.146:0/4175838982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 msgr2=0x7fa2cc0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 --2- 172.21.15.146:0/4175838982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc0ae290 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009f20 tx=0x7fa2c4009840 comp rx=0 tx=0).stop 2024-08-15T16:59:40.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- 172.21.15.146:0/4175838982 shutdown_connections 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 --2- 172.21.15.146:0/4175838982 >> v2:172.21.15.161:3300/0 conn(0x7fa2cc06e8c0 0x7fa2cc13f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 --2- 172.21.15.146:0/4175838982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc0ae290 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- 172.21.15.146:0/4175838982 >> 172.21.15.146:0/4175838982 conn(0x7fa2cc067880 msgr2=0x7fa2cc067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- 172.21.15.146:0/4175838982 shutdown_connections 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.501+0000 7fa2d3168700 1 -- 172.21.15.146:0/4175838982 wait complete. 2024-08-15T16:59:40.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 Processor -- start 2024-08-15T16:59:40.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 -- start start 2024-08-15T16:59:40.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 0x7fa2cc136ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2cc0aeb10 0x7fa2cc13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc1551a0 con 0x7fa2cc0adec0 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc136810 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d3168700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc135ee0 con 0x7fa2cc0aeb10 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d1965700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 0x7fa2cc136ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d1965700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50642/0 (socket says 172.21.15.146:50642) 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d1965700 1 -- 172.21.15.146:0/2923639419 learned_addr learned my addr 172.21.15.146:0/2923639419 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:40.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2967700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2cc0aeb10 0x7fa2cc13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 -- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2cc0aeb10 msgr2=0x7fa2cc13dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2cc0aeb10 0x7fa2cc13dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 -- 172.21.15.146:0/2923639419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 msgr2=0x7fa2cc063200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:40.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d2166700 1 -- 172.21.15.146:0/2923639419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4009ae0 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.505+0000 7fa2d1965700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2d2166700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 0x7fa2cc136ca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4015000 tx=0x7fa2c4018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c40088f0 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c4008a50 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc1361c0 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4008dd0 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc154e30 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc136020 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa2c4003040 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 0x7fa2b8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:40.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa2c407e620 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.509+0000 7fa2d1965700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 0x7fa2b8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:40.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.513+0000 7fa2d1965700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 0x7fa2b8062d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc159410 tx=0x7fa2c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:40.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.521+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa2cc136020 con 0x7fa2cc06e8c0 2024-08-15T16:59:40.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:40.685+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa2cc136020 con 0x7fa2cc06e8c0 2024-08-15T16:59:41.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.365+0000 7fa2ca7fc700 1 -- 172.21.15.146:0/2923639419 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa2cc136020 con 0x7fa2cc06e8c0 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 msgr2=0x7fa2b8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 0x7fa2b8062d10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc159410 tx=0x7fa2c0009040 comp rx=0 tx=0).stop 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 msgr2=0x7fa2cc136ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 0x7fa2cc136ca0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4015000 tx=0x7fa2c4018000 comp rx=0 tx=0).stop 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 shutdown_connections 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa2b8060860 0x7fa2b8062d10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa2cc0aeb10 0x7fa2cc13dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2cc0adec0 0x7fa2cc063200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 --2- 172.21.15.146:0/2923639419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa2cc06e8c0 0x7fa2cc136ca0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 >> 172.21.15.146:0/2923639419 conn(0x7fa2cc067880 msgr2=0x7fa2cc0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:41.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 shutdown_connections 2024-08-15T16:59:41.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.381+0000 7fa2d3168700 1 -- 172.21.15.146:0/2923639419 wait complete. 2024-08-15T16:59:41.396 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' created 2024-08-15T16:59:41.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T16:59:41.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.497+0000 7f923bfff700 1 Processor -- start 2024-08-15T16:59:41.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.497+0000 7f923bfff700 1 -- start start 2024-08-15T16:59:41.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f923c1590b0 0x7f923c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.795 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f923c0cd2b0 0x7f923c15ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0cba00 con 0x7f923c1590b0 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0cdc90 con 0x7f923c0cd2b0 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ce210 con 0x7f923c159480 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f923c1590b0 0x7f923c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f923c1590b0 0x7f923c066e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50646/0 (socket says 172.21.15.146:50646) 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 -- 172.21.15.146:0/3294930793 learned_addr learned my addr 172.21.15.146:0/3294930793 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923b7fe700 1 --2- 172.21.15.146:0/3294930793 >> v2:172.21.15.161:3300/0 conn(0x7f923c0cd2b0 0x7f923c15ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923a7fc700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 -- 172.21.15.146:0/3294930793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 msgr2=0x7f923c067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 -- 172.21.15.146:0/3294930793 >> v2:172.21.15.161:3300/0 conn(0x7f923c0cd2b0 msgr2=0x7f923c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:41.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 --2- 172.21.15.146:0/3294930793 >> v2:172.21.15.161:3300/0 conn(0x7f923c0cd2b0 0x7f923c15ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 -- 172.21.15.146:0/3294930793 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0cb440 con 0x7f923c1590b0 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923a7fc700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923affd700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c066e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f8d0 tx=0x7f9230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740f109f41f2afd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f9239ffb700 1 -- 172.21.15.146:0/3294930793 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230009e70 con 0x7f923c1590b0 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f9239ffb700 1 -- 172.21.15.146:0/3294930793 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923000b860 con 0x7f923c1590b0 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f9239ffb700 1 -- 172.21.15.146:0/3294930793 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923000bc00 con 0x7f923c1590b0 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- 172.21.15.146:0/3294930793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 msgr2=0x7f923c066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c066e00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f8d0 tx=0x7f9230009250 comp rx=0 tx=0).stop 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- 172.21.15.146:0/3294930793 shutdown_connections 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- 172.21.15.146:0/3294930793 >> v2:172.21.15.161:3300/0 conn(0x7f923c0cd2b0 0x7f923c15ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 --2- 172.21.15.146:0/3294930793 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c066e00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- 172.21.15.146:0/3294930793 >> 172.21.15.146:0/3294930793 conn(0x7f923c0d0b90 msgr2=0x7f923c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- 172.21.15.146:0/3294930793 shutdown_connections 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.501+0000 7f923bfff700 1 -- 172.21.15.146:0/3294930793 wait complete. 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 Processor -- start 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 -- start start 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f923c0cd2b0 0x7f923c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c155b00 con 0x7f923c1590b0 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0c72a0 con 0x7f923c0cd2b0 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923bfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c155ce0 con 0x7f923c159480 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50660/0 (socket says 172.21.15.146:50660) 2024-08-15T16:59:41.798 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 -- 172.21.15.146:0/356190402 learned_addr learned my addr 172.21.15.146:0/356190402 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923b7fe700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923affd700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f923c0cd2b0 0x7f923c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 -- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 msgr2=0x7f923c156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 -- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f923c0cd2b0 msgr2=0x7f923c143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f923c0cd2b0 0x7f923c143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.505+0000 7f923a7fc700 1 -- 172.21.15.146:0/356190402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92300094f0 con 0x7f923c1590b0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f923a7fc700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d860 tx=0x7f922c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c0125f0 con 0x7f923c1590b0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c012750 con 0x7f923c1590b0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c155fc0 con 0x7f923c1590b0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c012af0 con 0x7f923c1590b0 2024-08-15T16:59:41.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c156580 con 0x7f923c1590b0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c0c7430 con 0x7f923c1590b0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.509+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f922c013480 con 0x7f923c1590b0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.513+0000 7f9222ffd700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 0x7f9228062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.513+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f922c07cc60 con 0x7f923c1590b0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.513+0000 7f923affd700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 0x7f9228062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.513+0000 7f923affd700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 0x7f9228062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f8d0 tx=0x7f9230017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.517+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f922c04d110 con 0x7f923c1590b0 2024-08-15T16:59:41.800 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:41.721+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f923c064330 con 0x7f923c1590b0 2024-08-15T16:59:42.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.405+0000 7f9222ffd700 1 -- 172.21.15.146:0/356190402 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f922c050730 con 0x7f923c1590b0 2024-08-15T16:59:42.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 msgr2=0x7f9228062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 0x7f9228062d10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f8d0 tx=0x7f9230017000 comp rx=0 tx=0).stop 2024-08-15T16:59:42.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 msgr2=0x7f923c0c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d860 tx=0x7f922c008040 comp rx=0 tx=0).stop 2024-08-15T16:59:42.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 shutdown_connections 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9228060860 0x7f9228062d10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f923c159480 0x7f923c156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f923c1590b0 0x7f923c0c9670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 --2- 172.21.15.146:0/356190402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f923c0cd2b0 0x7f923c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.417+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 >> 172.21.15.146:0/356190402 conn(0x7f923c0d0b90 msgr2=0x7f923c063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.421+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 shutdown_connections 2024-08-15T16:59:42.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.421+0000 7f923bfff700 1 -- 172.21.15.146:0/356190402 wait complete. 2024-08-15T16:59:42.433 INFO:teuthology.orchestra.run.smithi146.stderr:new fs with metadata pool 2 and data pool 3 2024-08-15T16:59:42.443 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:42.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 Processor -- start 2024-08-15T16:59:42.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 -- start start 2024-08-15T16:59:42.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb6c0aa5f0 0x7fcb6c06ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0afc10 con 0x7fcb6c155e50 2024-08-15T16:59:42.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c0aed20 con 0x7fcb6c156220 2024-08-15T16:59:42.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.533+0000 7fcb70c6a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c0af000 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fcb6c155e50 0x7fcb6c0a9ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:34998/0 (socket says 172.21.15.146:34998) 2024-08-15T16:59:42.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6a59c700 1 -- 172.21.15.146:0/1463564998 learned_addr learned my addr 172.21.15.146:0/1463564998 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:42.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb69d9b700 1 --2- 172.21.15.146:0/1463564998 >> v2:172.21.15.146:3300/0 conn(0x7fcb6c0aa5f0 0x7fcb6c06ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024896520 0 0) 0x7fcb6c0afc10 con 0x7fcb6c155e50 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb54005780 con 0x7fcb6c155e50 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb69d9b700 1 -- 172.21.15.146:0/1463564998 >> v1:172.21.15.161:6789/0 conn(0x7fcb6c156220 legacy=0x7fcb6c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb69d9b700 1 -- 172.21.15.146:0/1463564998 >> v1:172.21.15.146:6789/0 conn(0x7fcb6c155e50 legacy=0x7fcb6c0a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 931269511 0 0) 0x7fcb54005780 con 0x7fcb6c155e50 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb69d9b700 1 -- 172.21.15.146:0/1463564998 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c0af2e0 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb69d9b700 1 --2- 172.21.15.146:0/1463564998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c0aa5f0 0x7fcb6c06ebe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb600098e0 tx=0x7fcb600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5178ed296e5224aa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640017f0 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb60011420 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb6959a700 1 -- 172.21.15.146:0/1463564998 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb600117c0 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 -- 172.21.15.146:0/1463564998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c0aa5f0 msgr2=0x7fcb6c06ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/1463564998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c0aa5f0 0x7fcb6c06ebe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb600098e0 tx=0x7fcb600091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 -- 172.21.15.146:0/1463564998 shutdown_connections 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/1463564998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c0aa5f0 0x7fcb6c06ebe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 -- 172.21.15.146:0/1463564998 >> 172.21.15.146:0/1463564998 conn(0x7fcb6c066e00 msgr2=0x7fcb6c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 -- 172.21.15.146:0/1463564998 shutdown_connections 2024-08-15T16:59:42.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.537+0000 7fcb70c6a700 1 -- 172.21.15.146:0/1463564998 wait complete. 2024-08-15T16:59:42.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 Processor -- start 2024-08-15T16:59:42.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- start start 2024-08-15T16:59:42.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb6c0aa5f0 0x7fcb6c1696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb6c155e50 0x7fcb6c098b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c097b30 con 0x7fcb6c156220 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c099db0 con 0x7fcb6c155e50 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c099b10 con 0x7fcb6c0aa5f0 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50686/0 (socket says 172.21.15.146:50686) 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 -- 172.21.15.146:0/668103516 learned_addr learned my addr 172.21.15.146:0/668103516 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6a59c700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb6c0aa5f0 0x7fcb6c1696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb69d9b700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb6c155e50 0x7fcb6c098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 -- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb6c0aa5f0 msgr2=0x7fcb6c1696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb6c0aa5f0 0x7fcb6c1696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 -- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb6c155e50 msgr2=0x7fcb6c098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb6c155e50 0x7fcb6c098b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 -- 172.21.15.146:0/668103516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009420 con 0x7fcb6c156220 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb6ad9d700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c560 tx=0x7fcb5c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011590 con 0x7fcb6c156220 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c0116f0 con 0x7fcb6c156220 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c097d60 con 0x7fcb6c156220 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c011ab0 con 0x7fcb6c156220 2024-08-15T16:59:42.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.541+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c098320 con 0x7fcb6c156220 2024-08-15T16:59:42.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.545+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcb5c01b3b0 con 0x7fcb6c156220 2024-08-15T16:59:42.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.545+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c0645c0 con 0x7fcb6c156220 2024-08-15T16:59:42.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.545+0000 7fcb5a7fc700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 0x7fcb4c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.545+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb5c07bf70 con 0x7fcb6c156220 2024-08-15T16:59:42.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.545+0000 7fcb6a59c700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 0x7fcb4c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.549+0000 7fcb6a59c700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 0x7fcb4c062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb64001560 tx=0x7fcb64000f40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.557+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcb5c04c3c0 con 0x7fcb6c156220 2024-08-15T16:59:42.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.721+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb6c0ae670 con 0x7fcb6c156220 2024-08-15T16:59:42.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.721+0000 7fcb5a7fc700 1 -- 172.21.15.146:0/668103516 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fcb5c017020 con 0x7fcb6c156220 2024-08-15T16:59:42.735 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:42.735 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:41.726541+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:42.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 msgr2=0x7fcb4c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 0x7fcb4c062d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb64001560 tx=0x7fcb64000f40 comp rx=0 tx=0).stop 2024-08-15T16:59:42.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 msgr2=0x7fcb6c0995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c560 tx=0x7fcb5c007ef0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.737 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 shutdown_connections 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb4c060860 0x7fcb4c062d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb6c156220 0x7fcb6c0995d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb6c155e50 0x7fcb6c098b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 --2- 172.21.15.146:0/668103516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb6c0aa5f0 0x7fcb6c1696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 >> 172.21.15.146:0/668103516 conn(0x7fcb6c066e00 msgr2=0x7fcb6c0ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 shutdown_connections 2024-08-15T16:59:42.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.725+0000 7fcb70c6a700 1 -- 172.21.15.146:0/668103516 wait complete. 2024-08-15T16:59:42.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 Processor -- start 2024-08-15T16:59:42.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- start start 2024-08-15T16:59:42.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 0x7fcab80af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcab815cea0 0x7fcab815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab80ca4c0 con 0x7fcab80aef40 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab80c95d0 con 0x7fcab815cea0 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab80c98b0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcab815cea0 0x7fcab815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcab815cea0 0x7fcab815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50698/0 (socket says 172.21.15.146:50698) 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 -- 172.21.15.146:0/397795563 learned_addr learned my addr 172.21.15.146:0/397795563 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab5d9b700 1 --2- 172.21.15.146:0/397795563 >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 0x7fcab80af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 -- 172.21.15.146:0/397795563 >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 msgr2=0x7fcab80af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 --2- 172.21.15.146:0/397795563 >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 0x7fcab80af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 -- 172.21.15.146:0/397795563 >> v1:172.21.15.161:6789/0 conn(0x7fcab80aef40 legacy=0x7fcab80adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 -- 172.21.15.146:0/397795563 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80c9b90 con 0x7fcab815cea0 2024-08-15T16:59:42.854 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab5d9b700 1 --2- 172.21.15.146:0/397795563 >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 0x7fcab80af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab6d9d700 1 --2- 172.21.15.146:0/397795563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab815cea0 0x7fcab815d320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab806d4c0 tx=0x7fcab0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8da5299c785636 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab559a700 1 -- 172.21.15.146:0/397795563 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000e070 con 0x7fcab815cea0 2024-08-15T16:59:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab559a700 1 -- 172.21.15.146:0/397795563 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab000ba10 con 0x7fcab815cea0 2024-08-15T16:59:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab559a700 1 -- 172.21.15.146:0/397795563 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00092e0 con 0x7fcab815cea0 2024-08-15T16:59:42.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- 172.21.15.146:0/397795563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab815cea0 msgr2=0x7fcab815d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 --2- 172.21.15.146:0/397795563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab815cea0 0x7fcab815d320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab806d4c0 tx=0x7fcab0009840 comp rx=0 tx=0).stop 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- 172.21.15.146:0/397795563 shutdown_connections 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 --2- 172.21.15.146:0/397795563 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab815cea0 0x7fcab815d320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 --2- 172.21.15.146:0/397795563 >> v2:172.21.15.161:3300/0 conn(0x7fcab80ae2c0 0x7fcab80af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- 172.21.15.146:0/397795563 >> 172.21.15.146:0/397795563 conn(0x7fcab806e070 msgr2=0x7fcab80b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- 172.21.15.146:0/397795563 shutdown_connections 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.841+0000 7fcab759e700 1 -- 172.21.15.146:0/397795563 wait complete. 2024-08-15T16:59:42.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 Processor -- start 2024-08-15T16:59:42.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 -- start start 2024-08-15T16:59:42.858 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab80aef40 0x7fcab806a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab815cea0 0x7fcab806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8161840 con 0x7fcab80ae2c0 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab8068020 con 0x7fcab815cea0 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab806b570 con 0x7fcab80aef40 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50702/0 (socket says 172.21.15.146:50702) 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab6d9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab815cea0 0x7fcab806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 -- 172.21.15.146:0/1702448410 learned_addr learned my addr 172.21.15.146:0/1702448410 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:42.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab5d9b700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab80aef40 0x7fcab806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 -- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab80aef40 msgr2=0x7fcab806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab80aef40 0x7fcab806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 -- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab815cea0 msgr2=0x7fcab806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab815cea0 0x7fcab806aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 -- 172.21.15.146:0/1702448410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009ae0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.845+0000 7fcab659c700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c5d0 tx=0x7fcaa8007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011590 con 0x7fcab80ae2c0 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa80116f0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab80679d0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011ab0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab806b850 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab80638f0 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.849+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcaa801c460 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.853+0000 7fcaa67fc700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 0x7fca98062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.853+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcaa807c770 con 0x7fcab80ae2c0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.853+0000 7fcab5d9b700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 0x7fca98062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:42.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.853+0000 7fcab5d9b700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 0x7fca98062de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab815c880 tx=0x7fcaac006f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:42.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:42.857+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcaa804cba0 con 0x7fcab80ae2c0 2024-08-15T16:59:43.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.021+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab80cc2a0 con 0x7fcab80ae2c0 2024-08-15T16:59:43.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.021+0000 7fcaa67fc700 1 -- 172.21.15.146:0/1702448410 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fcaa80501c0 con 0x7fcab80ae2c0 2024-08-15T16:59:43.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:43.035 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:41.726541+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 msgr2=0x7fca98062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 0x7fca98062de0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcab815c880 tx=0x7fcaac006f90 comp rx=0 tx=0).stop 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 msgr2=0x7fcab81557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c5d0 tx=0x7fcaa8007f60 comp rx=0 tx=0).stop 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 shutdown_connections 2024-08-15T16:59:43.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca98060930 0x7fca98062de0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcab815cea0 0x7fcab806aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcab80aef40 0x7fcab806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 --2- 172.21.15.146:0/1702448410 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab80ae2c0 0x7fcab81557c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 >> 172.21.15.146:0/1702448410 conn(0x7fcab806e070 msgr2=0x7fcab80ada30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 shutdown_connections 2024-08-15T16:59:43.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.025+0000 7fcab759e700 1 -- 172.21.15.146:0/1702448410 wait complete. 2024-08-15T16:59:43.048 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T16:59:43.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 Processor -- start 2024-08-15T16:59:43.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 -- start start 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f28341590b0 0x7f2834066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2834159480 0x7f2834067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28340ce210 con 0x7f28340cd2b0 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28340cba00 con 0x7f28341590b0 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.145+0000 7f28398fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28340cdc90 con 0x7f2834159480 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28390f9700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f28340cd2b0 0x7f283415ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35006/0 (socket says 172.21.15.146:35006) 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28390f9700 1 -- 172.21.15.146:0/2794344893 learned_addr learned my addr 172.21.15.146:0/2794344893 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 --2- 172.21.15.146:0/2794344893 >> v2:172.21.15.146:3300/0 conn(0x7f28341590b0 0x7f2834066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f2833fff700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2834159480 0x7f2834067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598934452 0 0) 0x7f28340ce210 con 0x7f28340cd2b0 2024-08-15T16:59:43.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f281c003570 con 0x7f28340cd2b0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 -- 172.21.15.146:0/2794344893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2834159480 msgr2=0x7f2834067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2834159480 0x7f2834067340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 -- 172.21.15.146:0/2794344893 >> v1:172.21.15.146:6789/0 conn(0x7f28340cd2b0 legacy=0x7f283415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722737567 0 0) 0x7f281c003570 con 0x7f28340cd2b0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 -- 172.21.15.146:0/2794344893 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340cb440 con 0x7f28341590b0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28388f8700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28341590b0 0x7f2834066e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28280098e0 tx=0x7f28280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28fff880d65ddc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828011630 con 0x7f28341590b0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2828011790 con 0x7f28341590b0 2024-08-15T16:59:43.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28337fe700 1 -- 172.21.15.146:0/2794344893 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828011b30 con 0x7f28341590b0 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 -- 172.21.15.146:0/2794344893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28341590b0 msgr2=0x7f2834066e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28341590b0 0x7f2834066e00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28280098e0 tx=0x7f28280091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 -- 172.21.15.146:0/2794344893 shutdown_connections 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2834159480 0x7f2834067340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 --2- 172.21.15.146:0/2794344893 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28341590b0 0x7f2834066e00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 -- 172.21.15.146:0/2794344893 >> 172.21.15.146:0/2794344893 conn(0x7f28340d0b90 msgr2=0x7f28340ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 -- 172.21.15.146:0/2794344893 shutdown_connections 2024-08-15T16:59:43.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.149+0000 7f28398fa700 1 -- 172.21.15.146:0/2794344893 wait complete. 2024-08-15T16:59:43.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 Processor -- start 2024-08-15T16:59:43.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- start start 2024-08-15T16:59:43.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28340cd2b0 0x7f2834151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28341590b0 0x7f283415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283416ce30 con 0x7f2834159480 2024-08-15T16:59:43.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283415e690 con 0x7f28340cd2b0 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283415da80 con 0x7f28341590b0 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50722/0 (socket says 172.21.15.146:50722) 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 -- 172.21.15.146:0/4150000735 learned_addr learned my addr 172.21.15.146:0/4150000735 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f2833fff700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28341590b0 0x7f283415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28388f8700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28340cd2b0 0x7f2834151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 -- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28341590b0 msgr2=0x7f283415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28341590b0 0x7f283415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 -- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28340cd2b0 msgr2=0x7f2834151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28340cd2b0 0x7f2834151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 -- 172.21.15.146:0/4150000735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828009420 con 0x7f2834159480 2024-08-15T16:59:43.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28390f9700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f282c00cb90 tx=0x7f282c007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0020f0 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c004dc0 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283415dd60 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c00a820 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.153+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283415e320 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28340675c0 con 0x7f2834159480 2024-08-15T16:59:43.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f282c00abd0 con 0x7f2834159480 2024-08-15T16:59:43.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f2830ff9700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 0x7f2820062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f282c07c510 con 0x7f2834159480 2024-08-15T16:59:43.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f28388f8700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 0x7f2820062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.157+0000 7f28388f8700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 0x7f2820062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28280091e0 tx=0x7f2828017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.165+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f28340675c0 con 0x7f2834159480 2024-08-15T16:59:43.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.369+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2834088010 con 0x7f2834159480 2024-08-15T16:59:43.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.405+0000 7f2830ff9700 1 -- 172.21.15.146:0/4150000735 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2834088010 con 0x7f2834159480 2024-08-15T16:59:43.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 msgr2=0x7f2820062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 0x7f2820062d10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28280091e0 tx=0x7f2828017000 comp rx=0 tx=0).stop 2024-08-15T16:59:43.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 msgr2=0x7f28340b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f282c00cb90 tx=0x7f282c007eb0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 shutdown_connections 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2820060860 0x7f2820062d10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2834159480 0x7f28340b4230 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28341590b0 0x7f283415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 --2- 172.21.15.146:0/4150000735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28340cd2b0 0x7f2834151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 >> 172.21.15.146:0/4150000735 conn(0x7f28340d0b90 msgr2=0x7f28340afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 shutdown_connections 2024-08-15T16:59:43.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.409+0000 7f28398fa700 1 -- 172.21.15.146:0/4150000735 wait complete. 2024-08-15T16:59:43.438 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T16:59:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 Processor -- start 2024-08-15T16:59:43.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 -- start start 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffac8067030 0x7ffac8067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffac80cfb00 0x7ffac80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80cf670 con 0x7ffac80cfb00 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac80cea60 con 0x7ffac8067030 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.541+0000 7ffacd87f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac80ced40 con 0x7ffac80cb5c0 2024-08-15T16:59:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffac80cfb00 0x7ffac80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffac80cfb00 0x7ffac80cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50734/0 (socket says 172.21.15.146:50734) 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 -- 172.21.15.146:0/2266821804 learned_addr learned my addr 172.21.15.146:0/2266821804 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacc87d700 1 --2- 172.21.15.146:0/2266821804 >> v2:172.21.15.161:3300/0 conn(0x7ffac8067030 0x7ffac8067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd07e700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 -- 172.21.15.146:0/2266821804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 msgr2=0x7ffac8158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 -- 172.21.15.146:0/2266821804 >> v2:172.21.15.161:3300/0 conn(0x7ffac8067030 msgr2=0x7ffac8067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 --2- 172.21.15.146:0/2266821804 >> v2:172.21.15.161:3300/0 conn(0x7ffac8067030 0x7ffac8067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 -- 172.21.15.146:0/2266821804 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80cf020 con 0x7ffac80cfb00 2024-08-15T16:59:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac7fff700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac80cb080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffac80d1070 tx=0x7ffab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5174ac9bc00f133f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac77fe700 1 -- 172.21.15.146:0/2266821804 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800b640 con 0x7ffac80cfb00 2024-08-15T16:59:43.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac77fe700 1 -- 172.21.15.146:0/2266821804 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800b7a0 con 0x7ffac80cfb00 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffac77fe700 1 -- 172.21.15.146:0/2266821804 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800bb60 con 0x7ffac80cfb00 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 -- 172.21.15.146:0/2266821804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 msgr2=0x7ffac80cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac80cb080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffac80d1070 tx=0x7ffab8009250 comp rx=0 tx=0).stop 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 -- 172.21.15.146:0/2266821804 shutdown_connections 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 --2- 172.21.15.146:0/2266821804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac80cb080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 --2- 172.21.15.146:0/2266821804 >> v2:172.21.15.161:3300/0 conn(0x7ffac8067030 0x7ffac8067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 -- 172.21.15.146:0/2266821804 >> 172.21.15.146:0/2266821804 conn(0x7ffac80d0b50 msgr2=0x7ffac80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 -- 172.21.15.146:0/2266821804 shutdown_connections 2024-08-15T16:59:43.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.545+0000 7ffacd87f700 1 -- 172.21.15.146:0/2266821804 wait complete. 2024-08-15T16:59:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 Processor -- start 2024-08-15T16:59:43.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- start start 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac8067030 0x7ffac816dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac8156f20 con 0x7ffac80cfb00 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac816d7f0 con 0x7ffac8067030 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac81569c0 con 0x7ffac80cb5c0 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacc87d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac8067030 0x7ffac816dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffac7fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50740/0 (socket says 172.21.15.146:50740) 2024-08-15T16:59:43.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 -- 172.21.15.146:0/78414363 learned_addr learned my addr 172.21.15.146:0/78414363 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 -- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 msgr2=0x7ffac8155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 -- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac8067030 msgr2=0x7ffac816dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac8067030 0x7ffac816dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 -- 172.21.15.146:0/78414363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab80094f0 con 0x7ffac80cfb00 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd07e700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009e90 tx=0x7ffac000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac000cb70 con 0x7ffac80cfb00 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac000f040 con 0x7ffac80cfb00 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac8156c40 con 0x7ffac80cfb00 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0014480 con 0x7ffac80cfb00 2024-08-15T16:59:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.549+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac816d480 con 0x7ffac80cfb00 2024-08-15T16:59:43.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac80638f0 con 0x7ffac80cfb00 2024-08-15T16:59:43.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffac0014830 con 0x7ffac80cfb00 2024-08-15T16:59:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffac4ff9700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 0x7ffab4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffac007d610 con 0x7ffac80cfb00 2024-08-15T16:59:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffacc87d700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 0x7ffab4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:43.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.553+0000 7ffacc87d700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 0x7ffab4062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009db0 tx=0x7ffabc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:43.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.561+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffac004da70 con 0x7ffac80cfb00 2024-08-15T16:59:43.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:43.765+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffac816d930 con 0x7ffac80cfb00 2024-08-15T16:59:44.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.413+0000 7ffac4ff9700 1 -- 172.21.15.146:0/78414363 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffac0051090 con 0x7ffac80cfb00 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 msgr2=0x7ffab4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 0x7ffab4062d10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009db0 tx=0x7ffabc0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 msgr2=0x7ffac8156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009e90 tx=0x7ffac000b040 comp rx=0 tx=0).stop 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 shutdown_connections 2024-08-15T16:59:44.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffab4060860 0x7ffab4062d10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffac80cfb00 0x7ffac8156480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffac80cb5c0 0x7ffac8155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 --2- 172.21.15.146:0/78414363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffac8067030 0x7ffac816dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 >> 172.21.15.146:0/78414363 conn(0x7ffac80d0b50 msgr2=0x7ffac80c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 shutdown_connections 2024-08-15T16:59:44.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.417+0000 7ffacd87f700 1 -- 172.21.15.146:0/78414363 wait complete. 2024-08-15T16:59:44.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:44.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 Processor -- start 2024-08-15T16:59:44.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 -- start start 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f27ac155d20 0x7f27ac0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27ac0ad7c0 0x7f27ac0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0af2b0 con 0x7f27ac06df10 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0adc10 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1b8a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0af590 con 0x7f27ac155d20 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1389700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f27ac06df10 0x7f27ac06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35020/0 (socket says 172.21.15.146:35020) 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27abfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f27ac0ad7c0 0x7f27ac0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b1389700 1 -- 172.21.15.146:0/1969065249 learned_addr learned my addr 172.21.15.146:0/1969065249 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:44.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.545+0000 7f27b0b88700 1 --2- 172.21.15.146:0/1969065249 >> v2:172.21.15.161:3300/0 conn(0x7f27ac155d20 0x7f27ac0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27abfff700 1 -- 172.21.15.146:0/1969065249 >> v2:172.21.15.161:3300/0 conn(0x7f27ac155d20 msgr2=0x7f27ac0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27abfff700 1 --2- 172.21.15.146:0/1969065249 >> v2:172.21.15.161:3300/0 conn(0x7f27ac155d20 0x7f27ac0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27abfff700 1 -- 172.21.15.146:0/1969065249 >> v1:172.21.15.146:6789/0 conn(0x7f27ac06df10 legacy=0x7f27ac06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27abfff700 1 -- 172.21.15.146:0/1969065249 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0af870 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27abfff700 1 --2- 172.21.15.146:0/1969065249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac0ad7c0 0x7f27ac0aea80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0b1d50 tx=0x7f27940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa371a0f665f12b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27ab7fe700 1 -- 172.21.15.146:0/1969065249 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400b6b0 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27ab7fe700 1 -- 172.21.15.146:0/1969065249 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279400b810 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27ab7fe700 1 -- 172.21.15.146:0/1969065249 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400bbd0 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- 172.21.15.146:0/1969065249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac0ad7c0 msgr2=0x7f27ac0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/1969065249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac0ad7c0 0x7f27ac0aea80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0b1d50 tx=0x7f27940091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- 172.21.15.146:0/1969065249 shutdown_connections 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/1969065249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac0ad7c0 0x7f27ac0aea80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/1969065249 >> v2:172.21.15.161:3300/0 conn(0x7f27ac155d20 0x7f27ac0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- 172.21.15.146:0/1969065249 >> 172.21.15.146:0/1969065249 conn(0x7f27ac0678d0 msgr2=0x7f27ac067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- 172.21.15.146:0/1969065249 shutdown_connections 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- 172.21.15.146:0/1969065249 wait complete. 2024-08-15T16:59:44.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 Processor -- start 2024-08-15T16:59:44.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.549+0000 7f27b1b8a700 1 -- start start 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27ac0ad7c0 0x7f27ac14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27ac155d20 0x7f27ac0ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac14f040 con 0x7f27ac06df10 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac14e430 con 0x7f27ac155d20 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac14e710 con 0x7f27ac0ad7c0 2024-08-15T16:59:44.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50762/0 (socket says 172.21.15.146:50762) 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 -- 172.21.15.146:0/676916520 learned_addr learned my addr 172.21.15.146:0/676916520 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1389700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27ac155d20 0x7f27ac0ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27abfff700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27ac0ad7c0 0x7f27ac14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 -- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27ac0ad7c0 msgr2=0x7f27ac14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27ac0ad7c0 0x7f27ac14dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 -- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27ac155d20 msgr2=0x7f27ac0ac0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27ac155d20 0x7f27ac0ac0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 -- 172.21.15.146:0/676916520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794009420 con 0x7f27ac06df10 2024-08-15T16:59:44.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b0b88700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f279c00d890 tx=0x7f279c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c0125f0 con 0x7f27ac06df10 2024-08-15T16:59:44.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c012750 con 0x7f27ac06df10 2024-08-15T16:59:44.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac14e9f0 con 0x7f27ac06df10 2024-08-15T16:59:44.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c012af0 con 0x7f27ac06df10 2024-08-15T16:59:44.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac158300 con 0x7f27ac06df10 2024-08-15T16:59:44.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.553+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0643a0 con 0x7f27ac06df10 2024-08-15T16:59:44.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.557+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f279c01e460 con 0x7f27ac06df10 2024-08-15T16:59:44.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.557+0000 7f27a8ff9700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 0x7f2790062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.557+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f279c07d0f0 con 0x7f27ac06df10 2024-08-15T16:59:44.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.557+0000 7f27abfff700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 0x7f2790062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.557+0000 7f27abfff700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 0x7f2790062d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27940091e0 tx=0x7f2794017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.561+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f279c04d540 con 0x7f27ac06df10 2024-08-15T16:59:44.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.757+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27ac06f5f0 con 0x7f27ac06df10 2024-08-15T16:59:44.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.757+0000 7f27a8ff9700 1 -- 172.21.15.146:0/676916520 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f279c050b60 con 0x7f27ac06df10 2024-08-15T16:59:44.770 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:44.770 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T16:59:44.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 msgr2=0x7f2790062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 0x7f2790062d40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27940091e0 tx=0x7f2794017000 comp rx=0 tx=0).stop 2024-08-15T16:59:44.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 msgr2=0x7f27ac13e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f279c00d890 tx=0x7f279c008040 comp rx=0 tx=0).stop 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 shutdown_connections 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2790060890 0x7f2790062d40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27ac155d20 0x7f27ac0ac0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27ac0ad7c0 0x7f27ac14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 --2- 172.21.15.146:0/676916520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f27ac06df10 0x7f27ac13e7b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 >> 172.21.15.146:0/676916520 conn(0x7f27ac0678d0 msgr2=0x7f27ac147a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 shutdown_connections 2024-08-15T16:59:44.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.761+0000 7f27b1b8a700 1 -- 172.21.15.146:0/676916520 wait complete. 2024-08-15T16:59:44.774 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T16:59:44.785 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 Processor -- start 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 -- start start 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9f8142860 0x7fc9f806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f806f2d0 0x7fc9f806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f80b2220 con 0x7fc9f8142c30 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f8159790 con 0x7fc9f8142860 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9feb21700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80b2500 con 0x7fc9f806f2d0 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc9f8142860 0x7fc9f806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fd31e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc9f8142c30 0x7fc9f80b0e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35034/0 (socket says 172.21.15.146:35034) 2024-08-15T16:59:44.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fd31e700 1 -- 172.21.15.146:0/2671480418 learned_addr learned my addr 172.21.15.146:0/2671480418 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fe320700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f806f2d0 0x7fc9f806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 -- 172.21.15.146:0/2671480418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f806f2d0 msgr2=0x7fc9f806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f806f2d0 0x7fc9f806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 -- 172.21.15.146:0/2671480418 >> v1:172.21.15.146:6789/0 conn(0x7fc9f8142c30 legacy=0x7fc9f80b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 -- 172.21.15.146:0/2671480418 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80b27e0 con 0x7fc9f8142860 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fdb1f700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f806e720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e60 tx=0x7fc9f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab4a14386be81d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fcb1d700 1 -- 172.21.15.146:0/2671480418 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000e070 con 0x7fc9f8142860 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fcb1d700 1 -- 172.21.15.146:0/2671480418 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000bc30 con 0x7fc9f8142860 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.877+0000 7fc9fcb1d700 1 -- 172.21.15.146:0/2671480418 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0012430 con 0x7fc9f8142860 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2671480418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 msgr2=0x7fc9f806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f806e720 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e60 tx=0x7fc9f0009780 comp rx=0 tx=0).stop 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2671480418 shutdown_connections 2024-08-15T16:59:44.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f806f2d0 0x7fc9f806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2671480418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f806e720 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2671480418 >> 172.21.15.146:0/2671480418 conn(0x7fc9f80ca2b0 msgr2=0x7fc9f806e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:44.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2671480418 shutdown_connections 2024-08-15T16:59:44.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2671480418 wait complete. 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 Processor -- start 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- start start 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f806f2d0 0x7fc9f8158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 0x7fc9f80c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c4940 con 0x7fc9f8142860 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c3d70 con 0x7fc9f806f2d0 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80c4010 con 0x7fc9f8142c30 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50780/0 (socket says 172.21.15.146:50780) 2024-08-15T16:59:44.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 -- 172.21.15.146:0/2756629692 learned_addr learned my addr 172.21.15.146:0/2756629692 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fdb1f700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f806f2d0 0x7fc9f8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fe320700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 0x7fc9f80c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 -- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 msgr2=0x7fc9f80c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 0x7fc9f80c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 -- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f806f2d0 msgr2=0x7fc9f8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f806f2d0 0x7fc9f8158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 -- 172.21.15.146:0/2756629692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0009a20 con 0x7fc9f8142860 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fe320700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 0x7fc9f80c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9fd31e700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c560 tx=0x7fc9e8001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8005b30 con 0x7fc9f8142860 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8005cb0 con 0x7fc9f8142860 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8007320 con 0x7fc9f8142860 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.881+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80c42f0 con 0x7fc9f8142860 2024-08-15T16:59:44.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f813e770 con 0x7fc9f8142860 2024-08-15T16:59:44.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8159530 con 0x7fc9f8142860 2024-08-15T16:59:44.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc9e8007690 con 0x7fc9f8142860 2024-08-15T16:59:44.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9edffb700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 0x7fc9dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:44.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc9e807bfc0 con 0x7fc9f8142860 2024-08-15T16:59:44.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9fdb1f700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 0x7fc9dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:44.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.885+0000 7fc9fdb1f700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 0x7fc9dc062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e60 tx=0x7fc9f0019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:44.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:44.893+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc9f8159530 con 0x7fc9f8142860 2024-08-15T16:59:45.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.053+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f8159530 con 0x7fc9f8142860 2024-08-15T16:59:45.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.053+0000 7fc9edffb700 1 -- 172.21.15.146:0/2756629692 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc9f8159530 con 0x7fc9f8142860 2024-08-15T16:59:45.066 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:45.066 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:45.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 msgr2=0x7fc9dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 0x7fc9dc062d10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009e60 tx=0x7fc9f0019000 comp rx=0 tx=0).stop 2024-08-15T16:59:45.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 msgr2=0x7fc9f80c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c560 tx=0x7fc9e8001020 comp rx=0 tx=0).stop 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 shutdown_connections 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc9dc060860 0x7fc9dc062d10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9f8142c30 0x7fc9f80c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9f8142860 0x7fc9f80c7e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 --2- 172.21.15.146:0/2756629692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9f806f2d0 0x7fc9f8158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 >> 172.21.15.146:0/2756629692 conn(0x7fc9f80ca2b0 msgr2=0x7fc9f80cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 shutdown_connections 2024-08-15T16:59:45.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.057+0000 7fc9feb21700 1 -- 172.21.15.146:0/2756629692 wait complete. 2024-08-15T16:59:45.081 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T16:59:45.081 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 Processor -- start 2024-08-15T16:59:45.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 -- start start 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbc94159040 0x7fbc940cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc940d0010 0x7fbc940d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc940cf190 con 0x7fbc9415cea0 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9415d9f0 con 0x7fbc94159040 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc93fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc940d09a0 con 0x7fbc940d0010 2024-08-15T16:59:45.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc92ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fbc94159040 0x7fbc940cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.173+0000 7fbc937fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fbc9415cea0 0x7fbc9415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35036/0 (socket says 172.21.15.146:35036) 2024-08-15T16:59:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc937fe700 1 -- 172.21.15.146:0/3032849719 learned_addr learned my addr 172.21.15.146:0/3032849719 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:45.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc927fc700 1 --2- 172.21.15.146:0/3032849719 >> v2:172.21.15.161:3300/0 conn(0x7fbc940d0010 0x7fbc940d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc91ffb700 1 -- 172.21.15.146:0/3032849719 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225427738 0 0) 0x7fbc940cf190 con 0x7fbc9415cea0 2024-08-15T16:59:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc91ffb700 1 -- 172.21.15.146:0/3032849719 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc80003570 con 0x7fbc9415cea0 2024-08-15T16:59:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc92ffd700 1 -- 172.21.15.146:0/3032849719 >> v2:172.21.15.161:3300/0 conn(0x7fbc940d0010 msgr2=0x7fbc940d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:45.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc92ffd700 1 --2- 172.21.15.146:0/3032849719 >> v2:172.21.15.161:3300/0 conn(0x7fbc940d0010 0x7fbc940d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc92ffd700 1 -- 172.21.15.146:0/3032849719 >> v1:172.21.15.146:6789/0 conn(0x7fbc9415cea0 legacy=0x7fbc9415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc92ffd700 1 -- 172.21.15.146:0/3032849719 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940ce5e0 con 0x7fbc94159040 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc92ffd700 1 --2- 172.21.15.146:0/3032849719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940cf620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc94067ef0 tx=0x7fbc840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3f2624c4c65559 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc91ffb700 1 -- 172.21.15.146:0/3032849719 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84011630 con 0x7fbc94159040 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc91ffb700 1 -- 172.21.15.146:0/3032849719 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc840117b0 con 0x7fbc94159040 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc91ffb700 1 -- 172.21.15.146:0/3032849719 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84011b70 con 0x7fbc94159040 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- 172.21.15.146:0/3032849719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 msgr2=0x7fbc940cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 --2- 172.21.15.146:0/3032849719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940cf620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc94067ef0 tx=0x7fbc840091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- 172.21.15.146:0/3032849719 shutdown_connections 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 --2- 172.21.15.146:0/3032849719 >> v2:172.21.15.161:3300/0 conn(0x7fbc940d0010 0x7fbc940d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 --2- 172.21.15.146:0/3032849719 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940cf620 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- 172.21.15.146:0/3032849719 >> 172.21.15.146:0/3032849719 conn(0x7fbc94068250 msgr2=0x7fbc94068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- 172.21.15.146:0/3032849719 shutdown_connections 2024-08-15T16:59:45.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- 172.21.15.146:0/3032849719 wait complete. 2024-08-15T16:59:45.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 Processor -- start 2024-08-15T16:59:45.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.177+0000 7fbc93fff700 1 -- start start 2024-08-15T16:59:45.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940d0010 0x7fbc940b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 0x7fbc9414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9414b200 con 0x7fbc94159040 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9414b390 con 0x7fbc940d0010 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940b0a10 con 0x7fbc9415cea0 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50786/0 (socket says 172.21.15.146:50786) 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 -- 172.21.15.146:0/1632154476 learned_addr learned my addr 172.21.15.146:0/1632154476 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc937fe700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 0x7fbc9414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc92ffd700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940d0010 0x7fbc940b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 -- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 msgr2=0x7fbc9414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 0x7fbc9414d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 -- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940d0010 msgr2=0x7fbc940b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940d0010 0x7fbc940b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 -- 172.21.15.146:0/1632154476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84009420 con 0x7fbc94159040 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc937fe700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 0x7fbc9414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc927fc700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e930 tx=0x7fbc7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c013680 con 0x7fbc94159040 2024-08-15T16:59:45.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0137e0 con 0x7fbc94159040 2024-08-15T16:59:45.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc940b0cf0 con 0x7fbc94159040 2024-08-15T16:59:45.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c013b80 con 0x7fbc94159040 2024-08-15T16:59:45.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc940b12b0 con 0x7fbc94159040 2024-08-15T16:59:45.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.181+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc940b2850 con 0x7fbc94159040 2024-08-15T16:59:45.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.185+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbc7c00c3e0 con 0x7fbc94159040 2024-08-15T16:59:45.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.185+0000 7fbc8affd700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 0x7fbc74062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.185+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc7c07d4f0 con 0x7fbc94159040 2024-08-15T16:59:45.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.185+0000 7fbc92ffd700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 0x7fbc74062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.185+0000 7fbc92ffd700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 0x7fbc74062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc94067ef0 tx=0x7fbc84017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.189+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbc7c04d9a0 con 0x7fbc94159040 2024-08-15T16:59:45.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.345+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc940643a0 con 0x7fbc74060860 2024-08-15T16:59:45.374 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.361+0000 7fbc8affd700 1 -- 172.21.15.146:0/1632154476 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253849 (secure 0 0 0) 0x7fbc940643a0 con 0x7fbc74060860 2024-08-15T16:59:45.381 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 msgr2=0x7fbc74062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 0x7fbc74062d10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbc94067ef0 tx=0x7fbc84017000 comp rx=0 tx=0).stop 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 msgr2=0x7fbc940b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e930 tx=0x7fbc7c0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 shutdown_connections 2024-08-15T16:59:45.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbc74060860 0x7fbc74062d10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc9415cea0 0x7fbc9414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbc94159040 0x7fbc940b22b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 --2- 172.21.15.146:0/1632154476 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc940d0010 0x7fbc940b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 >> 172.21.15.146:0/1632154476 conn(0x7fbc94068250 msgr2=0x7fbc940b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 shutdown_connections 2024-08-15T16:59:45.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.369+0000 7fbc93fff700 1 -- 172.21.15.146:0/1632154476 wait complete. 2024-08-15T16:59:45.387 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:45.393 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-08-15T16:59:43.643616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385796,"kb_used_data":2472,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588924,"statfs":{"total":773094113280,"available":770651058176,"internally_reserved":0,"allocated":2531328,"data_stored":1586550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.287876"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363782+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367602+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363667+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356727+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366544+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357002+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370005+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367198+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370857+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:46.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.365523+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:34:49.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366399+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373506+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.364080+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373181+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369153+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373285+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-08-15T16:59:39.475362+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:39.475362+0000","last_peered":"2024-08-15T16:59:39.475362+0000","last_clean":"2024-08-15T16:59:39.475362+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:39.475362+0000","last_undegraded":"2024-08-15T16:59:39.475362+0000","last_fullsized":"2024-08-15T16:59:39.475362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357832+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373221+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356721+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369228+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.359423+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369653+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.358937+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369167+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.368697+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357835+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357941+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.360200+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363800+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.371247+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":757760,"data_stored":639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.96}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":753664,"data_stored":638077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:45.398 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:45.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 Processor -- start 2024-08-15T16:59:45.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 -- start start 2024-08-15T16:59:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0c4153a40 0x7fb0c40c9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c415a260 0x7fb0c40ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c40c9e00 con 0x7fb0c40b29f0 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40ccd20 con 0x7fb0c4153a40 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c947a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cc890 con 0x7fb0c415a260 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c27fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb0c40b29f0 0x7fb0c4159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35040/0 (socket says 172.21.15.146:35040) 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c27fc700 1 -- 172.21.15.146:0/3493025510 learned_addr learned my addr 172.21.15.146:0/3493025510 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3493025510 >> v2:172.21.15.161:3300/0 conn(0x7fb0c4153a40 0x7fb0c40c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c37fe700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c415a260 0x7fb0c40ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c1ffb700 1 -- 172.21.15.146:0/3493025510 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426252677 0 0) 0x7fb0c40c9e00 con 0x7fb0c40b29f0 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c1ffb700 1 -- 172.21.15.146:0/3493025510 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ac003570 con 0x7fb0c40b29f0 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 -- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c415a260 msgr2=0x7fb0c40ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c415a260 0x7fb0c40ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 -- 172.21.15.146:0/3493025510 >> v1:172.21.15.146:6789/0 conn(0x7fb0c40b29f0 legacy=0x7fb0c4159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 -- 172.21.15.146:0/3493025510 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40cbf60 con 0x7fb0c4153a40 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c40c9820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80098e0 tx=0x7fb0b80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e0b748b0c16953 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c1ffb700 1 -- 172.21.15.146:0/3493025510 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8011630 con 0x7fb0c4153a40 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c1ffb700 1 -- 172.21.15.146:0/3493025510 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b8011790 con 0x7fb0c4153a40 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.497+0000 7fb0c1ffb700 1 -- 172.21.15.146:0/3493025510 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8011b30 con 0x7fb0c4153a40 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 msgr2=0x7fb0c40c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c40c9820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80098e0 tx=0x7fb0b80091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- 172.21.15.146:0/3493025510 shutdown_connections 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c415a260 0x7fb0c40ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3493025510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c40c9820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- 172.21.15.146:0/3493025510 >> 172.21.15.146:0/3493025510 conn(0x7fb0c40a9db0 msgr2=0x7fb0c40ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- 172.21.15.146:0/3493025510 shutdown_connections 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- 172.21.15.146:0/3493025510 wait complete. 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 Processor -- start 2024-08-15T16:59:45.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- start start 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c40b29f0 0x7fb0c40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c4098800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4167da0 con 0x7fb0c415a260 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4167200 con 0x7fb0c4153a40 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c947a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4167470 con 0x7fb0c40b29f0 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50790/0 (socket says 172.21.15.146:50790) 2024-08-15T16:59:45.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c37fe700 1 -- 172.21.15.146:0/3403633655 learned_addr learned my addr 172.21.15.146:0/3403633655 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:45.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c27fc700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c4098800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.501+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c40b29f0 0x7fb0c40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 -- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c40b29f0 msgr2=0x7fb0c40c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c40b29f0 0x7fb0c40c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 -- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 msgr2=0x7fb0c4098800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c4098800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 -- 172.21.15.146:0/3403633655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8009420 con 0x7fb0c415a260 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c37fe700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400ea50 tx=0x7fb0b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400ee90 con 0x7fb0c415a260 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400c850 con 0x7fb0c415a260 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4167750 con 0x7fb0c415a260 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400cc10 con 0x7fb0c415a260 2024-08-15T16:59:45.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c40c9970 con 0x7fb0c415a260 2024-08-15T16:59:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c4167f50 con 0x7fb0c415a260 2024-08-15T16:59:45.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb0b4013470 con 0x7fb0c415a260 2024-08-15T16:59:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 0x7fb0b0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.505+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0b407cd90 con 0x7fb0c415a260 2024-08-15T16:59:45.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.509+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 0x7fb0b0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.509+0000 7fb0c2ffd700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 0x7fb0b0062d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80091e0 tx=0x7fb0b8017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.513+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb0c4167f50 con 0x7fb0c415a260 2024-08-15T16:59:45.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.673+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0c40c9bf0 con 0x7fb0b0060890 2024-08-15T16:59:45.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.689+0000 7fb0aaffd700 1 -- 172.21.15.146:0/3403633655 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253849 (secure 0 0 0) 0x7fb0c40c9bf0 con 0x7fb0b0060890 2024-08-15T16:59:45.705 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:45.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 msgr2=0x7fb0b0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 0x7fb0b0062d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80091e0 tx=0x7fb0b8017000 comp rx=0 tx=0).stop 2024-08-15T16:59:45.707 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 msgr2=0x7fb0c4098d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400ea50 tx=0x7fb0b40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 shutdown_connections 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb0b0060890 0x7fb0b0062d40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb0c415a260 0x7fb0c4098d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0c4153a40 0x7fb0c4098800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 --2- 172.21.15.146:0/3403633655 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0c40b29f0 0x7fb0c40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 >> 172.21.15.146:0/3403633655 conn(0x7fb0c40a9db0 msgr2=0x7fb0c40aba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 shutdown_connections 2024-08-15T16:59:45.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.693+0000 7fb0c947a700 1 -- 172.21.15.146:0/3403633655 wait complete. 2024-08-15T16:59:45.709 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:45.723 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-08-15T16:59:45.644287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385796,"kb_used_data":2472,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588924,"statfs":{"total":773094113280,"available":770651058176,"internally_reserved":0,"allocated":2531328,"data_stored":1586550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.288547"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363782+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367602+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363667+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356727+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366544+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357002+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370005+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367198+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370857+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:46.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.365523+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:34:49.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366399+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373506+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.364080+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373181+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369153+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373285+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-08-15T16:59:39.475362+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:39.475362+0000","last_peered":"2024-08-15T16:59:39.475362+0000","last_clean":"2024-08-15T16:59:39.475362+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:39.475362+0000","last_undegraded":"2024-08-15T16:59:39.475362+0000","last_fullsized":"2024-08-15T16:59:39.475362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357832+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373221+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356721+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369228+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.359423+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369653+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.358937+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369167+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.368697+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357835+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357941+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.360200+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363800+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.371247+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":757760,"data_stored":639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.96}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":753664,"data_stored":638077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:45.729 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:45.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 Processor -- start 2024-08-15T16:59:45.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- start start 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c48153a50 0x7f0c480687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0c4806da00 0x7f0c480afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4815ce60 con 0x7f0c480ae000 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4815d0b0 con 0x7f0c48153a50 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4815d390 con 0x7f0c4806da00 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c48153a50 0x7f0c480687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0c48153a50 0x7f0c480687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50798/0 (socket says 172.21.15.146:50798) 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 -- 172.21.15.146:0/228369101 learned_addr learned my addr 172.21.15.146:0/228369101 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:45.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c477fe700 1 --2- 172.21.15.146:0/228369101 >> v2:172.21.15.161:3300/0 conn(0x7f0c4806da00 0x7f0c480afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 -- 172.21.15.146:0/228369101 >> v2:172.21.15.161:3300/0 conn(0x7f0c4806da00 msgr2=0x7f0c480afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 --2- 172.21.15.146:0/228369101 >> v2:172.21.15.161:3300/0 conn(0x7f0c4806da00 0x7f0c480afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 -- 172.21.15.146:0/228369101 >> v1:172.21.15.161:6789/0 conn(0x7f0c480ae000 legacy=0x7f0c4806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 -- 172.21.15.146:0/228369101 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4815d670 con 0x7f0c48153a50 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c46ffd700 1 --2- 172.21.15.146:0/228369101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c48153a50 0x7f0c480687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c48153e20 tx=0x7f0c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650eff02e2d5ae20 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c467fc700 1 -- 172.21.15.146:0/228369101 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c38009e70 con 0x7f0c48153a50 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c467fc700 1 -- 172.21.15.146:0/228369101 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3800b860 con 0x7f0c48153a50 2024-08-15T16:59:45.843 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c467fc700 1 -- 172.21.15.146:0/228369101 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800bc00 con 0x7f0c48153a50 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/228369101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c48153a50 msgr2=0x7f0c480687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/228369101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c48153a50 0x7f0c480687a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c48153e20 tx=0x7f0c38009250 comp rx=0 tx=0).stop 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/228369101 shutdown_connections 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/228369101 >> v2:172.21.15.161:3300/0 conn(0x7f0c4806da00 0x7f0c480afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/228369101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c48153a50 0x7f0c480687a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/228369101 >> 172.21.15.146:0/228369101 conn(0x7f0c480ceb30 msgr2=0x7f0c4806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/228369101 shutdown_connections 2024-08-15T16:59:45.844 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.829+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/228369101 wait complete. 2024-08-15T16:59:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 Processor -- start 2024-08-15T16:59:45.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 -- start start 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c4806da00 0x7f0c480c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 0x7f0c480c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48152f20 con 0x7f0c480ae000 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480c7370 con 0x7f0c480c9510 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c4d9a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4816aae0 con 0x7f0c4806da00 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c3e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c3e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50812/0 (socket says 172.21.15.146:50812) 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c3e7fc700 1 -- 172.21.15.146:0/778395133 learned_addr learned my addr 172.21.15.146:0/778395133 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c46ffd700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c4806da00 0x7f0c480c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c477fe700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 0x7f0c480c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c3e7fc700 1 -- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c4806da00 msgr2=0x7f0c480c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.833+0000 7f0c3e7fc700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c4806da00 0x7f0c480c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3e7fc700 1 -- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 msgr2=0x7f0c480c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3e7fc700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 0x7f0c480c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3e7fc700 1 -- 172.21.15.146:0/778395133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380094f0 con 0x7f0c480ae000 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c477fe700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 0x7f0c480c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3e7fc700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c4e0 tx=0x7f0c3000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011590 con 0x7f0c480ae000 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c30011710 con 0x7f0c480ae000 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c480c7610 con 0x7f0c480ae000 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30011ad0 con 0x7f0c480ae000 2024-08-15T16:59:45.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c48157940 con 0x7f0c480ae000 2024-08-15T16:59:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c480c7800 con 0x7f0c480ae000 2024-08-15T16:59:45.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0c3001b3b0 con 0x7f0c480ae000 2024-08-15T16:59:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 0x7f0c28062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.837+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c3007bfa0 con 0x7f0c480ae000 2024-08-15T16:59:45.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.841+0000 7f0c46ffd700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 0x7f0c28062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:45.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.841+0000 7f0c46ffd700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 0x7f0c28062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c48153e20 tx=0x7f0c38017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:45.857 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:45.845+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0c3004c3f0 con 0x7f0c480ae000 2024-08-15T16:59:46.014 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.001+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c480645c0 con 0x7f0c28060860 2024-08-15T16:59:46.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.017+0000 7f0c3f7fe700 1 -- 172.21.15.146:0/778395133 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253849 (secure 0 0 0) 0x7f0c480645c0 con 0x7f0c28060860 2024-08-15T16:59:46.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:46.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 msgr2=0x7f0c28062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 0x7f0c28062d10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0c48153e20 tx=0x7f0c38017000 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 msgr2=0x7f0c48157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000c4e0 tx=0x7f0c3000c7f0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 shutdown_connections 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0c28060860 0x7f0c28062d10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0c480c9510 0x7f0c480c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c480ae000 0x7f0c48157400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 --2- 172.21.15.146:0/778395133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0c4806da00 0x7f0c480c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 >> 172.21.15.146:0/778395133 conn(0x7f0c480ceb30 msgr2=0x7f0c4815b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 shutdown_connections 2024-08-15T16:59:46.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.025+0000 7f0c4d9a7700 1 -- 172.21.15.146:0/778395133 wait complete. 2024-08-15T16:59:46.038 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:46.051 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-08-15T16:59:45.644287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385796,"kb_used_data":2472,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588924,"statfs":{"total":773094113280,"available":770651058176,"internally_reserved":0,"allocated":2531328,"data_stored":1586550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.288547"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363782+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367602+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363667+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356727+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366544+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357002+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370005+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367198+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370857+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:46.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.365523+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:34:49.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366399+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373506+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.364080+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373181+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369153+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373285+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-08-15T16:59:39.475362+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:39.475362+0000","last_peered":"2024-08-15T16:59:39.475362+0000","last_clean":"2024-08-15T16:59:39.475362+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:39.475362+0000","last_undegraded":"2024-08-15T16:59:39.475362+0000","last_fullsized":"2024-08-15T16:59:39.475362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357832+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373221+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356721+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369228+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.359423+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369653+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.358937+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369167+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.368697+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357835+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357941+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.360200+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363800+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.371247+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":757760,"data_stored":639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.96}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":753664,"data_stored":638077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-08-15T16:59:46.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-08-15T16:59:46.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-08-15T16:59:46.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-08-15T16:59:46.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-08-15T16:59:46.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-08-15T16:59:46.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-08-15T16:59:46.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-08-15T16:59:46.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-08-15T16:59:46.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-08-15T16:59:46.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-08-15T16:59:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-08-15T16:59:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-08-15T16:59:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-08-15T16:59:46.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-08-15T16:59:46.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-08-15T16:59:46.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-08-15T16:59:46.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-08-15T16:59:46.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-08-15T16:59:46.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-08-15T16:59:46.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-08-15T16:59:46.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-08-15T16:59:46.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-08-15T16:59:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.363782+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T16:59:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-08-15T16:59:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-08-15T16:59:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.367602+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T16:59:46.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-08-15T16:59:46.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.363667+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T16:59:46.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-08-15T16:59:46.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-08-15T16:59:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.356727+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T16:59:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-08-15T16:59:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-08-15T16:59:46.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.366544+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T16:59:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-08-15T16:59:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-08-15T16:59:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.357002+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T16:59:46.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-08-15T16:59:46.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-08-15T16:59:46.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.370005+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-08-15T16:59:46.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-08-15T16:59:46.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.367198+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T16:59:46.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-08-15T16:59:46.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.370857+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T03:21:46.205676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.365523+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T20:34:49.543171+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T16:59:46.073 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.366399+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.373506+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-08-15T16:59:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.364080+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.373181+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.369153+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-08-15T16:59:46.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.373285+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.357021+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-08-15T16:59:46.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.357832+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.373221+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.356721+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-08-15T16:59:46.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.369228+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.359423+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-08-15T16:59:46.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.369653+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.366021+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.358937+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-08-15T16:59:46.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.369167+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.368697+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-08-15T16:59:46.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.357835+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.357941+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.360200+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-08-15T16:59:46.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.363800+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-08-15T16:59:41.349871+0000', 'last_change': '2024-08-15T16:59:41.349871+0000', 'last_active': '2024-08-15T16:59:41.349871+0000', 'last_peered': '2024-08-15T16:59:41.349871+0000', 'last_clean': '2024-08-15T16:59:41.349871+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:41.349871+0000', 'last_undegraded': '2024-08-15T16:59:41.349871+0000', 'last_fullsized': '2024-08-15T16:59:41.349871+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:41.349871+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-08-15T16:59:46.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2024-08-15T16:59:40.339920+0000', 'last_change': '2024-08-15T16:59:40.371247+0000', 'last_active': '2024-08-15T16:59:40.339920+0000', 'last_peered': '2024-08-15T16:59:40.339920+0000', 'last_clean': '2024-08-15T16:59:40.339920+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-08-15T16:59:40.339920+0000', 'last_undegraded': '2024-08-15T16:59:40.339920+0000', 'last_fullsized': '2024-08-15T16:59:40.339920+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'last_clean_scrub_stamp': '2024-08-15T16:59:40.339920+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-08-15T16:59:46.083 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-08-15T16:59:46.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 Processor -- start 2024-08-15T16:59:46.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 -- start start 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff238156220 0x7ff238160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2380afc10 con 0x7ff238155e50 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2380b10b0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff240397700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2380aed20 con 0x7ff238156220 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff238156220 0x7ff238160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff238156220 0x7ff238160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50814/0 (socket says 172.21.15.146:50814) 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 -- 172.21.15.146:0/292012287 learned_addr learned my addr 172.21.15.146:0/292012287 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e133700 1 --1- 172.21.15.146:0/292012287 >> v1:172.21.15.146:6789/0 conn(0x7ff238155e50 0x7ff2380a9ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:35050/0 (socket says 172.21.15.146:35050) 2024-08-15T16:59:46.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176715307 0 0) 0x7ff2380b10b0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff224003570 con 0x7ff2380aa5f0 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 -- 172.21.15.146:0/292012287 >> v1:172.21.15.161:6789/0 conn(0x7ff2380aa5f0 legacy=0x7ff23806ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 -- 172.21.15.146:0/292012287 >> v1:172.21.15.146:6789/0 conn(0x7ff238155e50 legacy=0x7ff2380a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810925981 0 0) 0x7ff2380afc10 con 0x7ff238155e50 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 -- 172.21.15.146:0/292012287 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2380af000 con 0x7ff238156220 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23e934700 1 --2- 172.21.15.146:0/292012287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff238156220 0x7ff238160f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2300099b0 tx=0x7ff230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db21c8ed21990e6b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.181+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230009ea0 con 0x7ff238156220 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000b860 con 0x7ff238156220 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff23d131700 1 -- 172.21.15.146:0/292012287 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000bbe0 con 0x7ff238156220 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- 172.21.15.146:0/292012287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff238156220 msgr2=0x7ff238160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 --2- 172.21.15.146:0/292012287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff238156220 0x7ff238160f50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2300099b0 tx=0x7ff230009250 comp rx=0 tx=0).stop 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- 172.21.15.146:0/292012287 shutdown_connections 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 --2- 172.21.15.146:0/292012287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff238156220 0x7ff238160f50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- 172.21.15.146:0/292012287 >> 172.21.15.146:0/292012287 conn(0x7ff238066e00 msgr2=0x7ff238067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- 172.21.15.146:0/292012287 shutdown_connections 2024-08-15T16:59:46.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- 172.21.15.146:0/292012287 wait complete. 2024-08-15T16:59:46.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 Processor -- start 2024-08-15T16:59:46.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.185+0000 7ff240397700 1 -- start start 2024-08-15T16:59:46.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff238155e50 0x7ff238098f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff238156220 0x7ff238099470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238098210 con 0x7ff2380aa5f0 2024-08-15T16:59:46.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380c9de0 con 0x7ff238156220 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff238097600 con 0x7ff238155e50 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50828/0 (socket says 172.21.15.146:50828) 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 -- 172.21.15.146:0/865641244 learned_addr learned my addr 172.21.15.146:0/865641244 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e934700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff238156220 0x7ff238099470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23d932700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff238155e50 0x7ff238098f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 -- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff238155e50 msgr2=0x7ff238098f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff238155e50 0x7ff238098f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 -- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff238156220 msgr2=0x7ff238099470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff238156220 0x7ff238099470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 -- 172.21.15.146:0/865641244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2300094f0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff23e133700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2280011e0 tx=0x7ff228007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228005a70 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228005bd0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2380978e0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2280070d0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.189+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff238097ea0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2380645c0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff228007480 con 0x7ff2380aa5f0 2024-08-15T16:59:46.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff2367fc700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 0x7ff218062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff22807bfb0 con 0x7ff2380aa5f0 2024-08-15T16:59:46.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff23d932700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 0x7ff218062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.193+0000 7ff23d932700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 0x7ff218062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff23814c110 tx=0x7ff220002720 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.201+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff22804a480 con 0x7ff2380aa5f0 2024-08-15T16:59:46.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.421+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff2380ae670 con 0x7ff2380aa5f0 2024-08-15T16:59:46.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.421+0000 7ff2367fc700 1 -- 172.21.15.146:0/865641244 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1109 (secure 0 0 0) 0x7ff228017080 con 0x7ff2380aa5f0 2024-08-15T16:59:46.435 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:46.435 INFO:teuthology.orchestra.run.smithi146.stdout:{"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1723741175,"num_in_osds":8,"osd_in_since":1723741167,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2443055104,"bytes_avail":770651058176,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4245}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-08-15T16:59:25.325294+0000","services":{}},"progress_events":{}} 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 msgr2=0x7ff218062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 0x7ff218062d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff23814c110 tx=0x7ff220002720 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 msgr2=0x7ff23814b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2280011e0 tx=0x7ff228007e10 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 shutdown_connections 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff218060860 0x7ff218062d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff238156220 0x7ff238099470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff238155e50 0x7ff238098f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 --2- 172.21.15.146:0/865641244 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff2380aa5f0 0x7ff23814b650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 >> 172.21.15.146:0/865641244 conn(0x7ff238066e00 msgr2=0x7ff238154820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 shutdown_connections 2024-08-15T16:59:46.438 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.425+0000 7ff240397700 1 -- 172.21.15.146:0/865641244 wait complete. 2024-08-15T16:59:46.449 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-08-15T16:59:46.449 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:46.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.541+0000 7f5c974cc700 1 Processor -- start 2024-08-15T16:59:46.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.541+0000 7f5c974cc700 1 -- start start 2024-08-15T16:59:46.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 0x7f5c900b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c900c86e0 con 0x7f5c9015cd90 2024-08-15T16:59:46.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9015d210 con 0x7f5c9015c620 2024-08-15T16:59:46.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900cb850 con 0x7f5c900ad1b0 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c964ca700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 0x7f5c900b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c95cc9700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5c9015cd90 0x7f5c900acaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:43660/0 (socket says 172.21.15.146:43660) 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c95cc9700 1 -- 172.21.15.146:0/1221760710 learned_addr learned my addr 172.21.15.146:0/1221760710 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:56798/0 (socket says 172.21.15.146:56798) 2024-08-15T16:59:46.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 -- 172.21.15.146:0/1221760710 >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 msgr2=0x7f5c900b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 --2- 172.21.15.146:0/1221760710 >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 0x7f5c900b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 -- 172.21.15.146:0/1221760710 >> v1:172.21.15.161:6789/0 conn(0x7f5c9015cd90 legacy=0x7f5c900acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 -- 172.21.15.146:0/1221760710 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9015d490 con 0x7f5c900ad1b0 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c964ca700 1 --2- 172.21.15.146:0/1221760710 >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 0x7f5c900b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c96ccb700 1 --2- 172.21.15.146:0/1221760710 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009e60 tx=0x7f5c88009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6781a28d41da546 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c954c8700 1 -- 172.21.15.146:0/1221760710 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800e070 con 0x7f5c900ad1b0 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c954c8700 1 -- 172.21.15.146:0/1221760710 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8800bb60 con 0x7f5c900ad1b0 2024-08-15T16:59:46.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c954c8700 1 -- 172.21.15.146:0/1221760710 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88012430 con 0x7f5c900ad1b0 2024-08-15T16:59:46.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- 172.21.15.146:0/1221760710 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 msgr2=0x7f5c900c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1221760710 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5c88009e60 tx=0x7f5c88009780 comp rx=0 tx=0).stop 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- 172.21.15.146:0/1221760710 shutdown_connections 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1221760710 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c900ad1b0 0x7f5c900c8b70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1221760710 >> v2:172.21.15.161:3300/0 conn(0x7f5c9015c620 0x7f5c900b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- 172.21.15.146:0/1221760710 >> 172.21.15.146:0/1221760710 conn(0x7f5c900ca900 msgr2=0x7f5c900cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- 172.21.15.146:0/1221760710 shutdown_connections 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.545+0000 7f5c974cc700 1 -- 172.21.15.146:0/1221760710 wait complete. 2024-08-15T16:59:46.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 Processor -- start 2024-08-15T16:59:46.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- start start 2024-08-15T16:59:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c9015c620 0x7f5c900b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c9015cd90 0x7f5c9016bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900aade0 con 0x7f5c900ad1b0 2024-08-15T16:59:46.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900aa1d0 con 0x7f5c9015cd90 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900aa4b0 con 0x7f5c9015c620 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50834/0 (socket says 172.21.15.146:50834) 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 -- 172.21.15.146:0/1412972493 learned_addr learned my addr 172.21.15.146:0/1412972493 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c96ccb700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c9015cd90 0x7f5c9016bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c95cc9700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c9015c620 0x7f5c900b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 -- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c9015c620 msgr2=0x7f5c900b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c9015c620 0x7f5c900b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 -- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c9015cd90 msgr2=0x7f5c9016bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c9015cd90 0x7f5c9016bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 -- 172.21.15.146:0/1412972493 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009420 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c964ca700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00ec80 tx=0x7f5c8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00c920 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00ca80 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c88009a20 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00ce00 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.549+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c900aaaf0 con 0x7f5c900ad1b0 2024-08-15T16:59:46.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c900643a0 con 0x7f5c900ad1b0 2024-08-15T16:59:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5c8c013680 con 0x7f5c900ad1b0 2024-08-15T16:59:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c867fc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 0x7f5c74062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c8c07cff0 con 0x7f5c900ad1b0 2024-08-15T16:59:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c95cc9700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 0x7f5c74062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:46.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.553+0000 7f5c95cc9700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 0x7f5c74062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c9006d7c0 tx=0x7f5c80009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:46.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.561+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5c8c04d3a0 con 0x7f5c900ad1b0 2024-08-15T16:59:46.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.721+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c900aaf20 con 0x7f5c74060860 2024-08-15T16:59:46.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.737+0000 7f5c867fc700 1 -- 172.21.15.146:0/1412972493 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253849 (secure 0 0 0) 0x7f5c800015e0 con 0x7f5c74060860 2024-08-15T16:59:46.754 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:46.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 msgr2=0x7f5c74062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 0x7f5c74062d10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5c9006d7c0 tx=0x7f5c80009040 comp rx=0 tx=0).stop 2024-08-15T16:59:46.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 msgr2=0x7f5c900a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00ec80 tx=0x7f5c8c0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 shutdown_connections 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5c74060860 0x7f5c74062d10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c9015cd90 0x7f5c9016bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c9015c620 0x7f5c900b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 --2- 172.21.15.146:0/1412972493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5c900ad1b0 0x7f5c900a9860 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 >> 172.21.15.146:0/1412972493 conn(0x7f5c900ca900 msgr2=0x7f5c900c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 shutdown_connections 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:46.741+0000 7f5c974cc700 1 -- 172.21.15.146:0/1412972493 wait complete. 2024-08-15T16:59:46.757 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:46.767 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-08-15T16:59:45.644287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":1,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385796,"kb_used_data":2472,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588924,"statfs":{"total":773094113280,"available":770651058176,"internally_reserved":0,"allocated":2531328,"data_stored":1586550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.288547"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363782+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367602+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363667+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356727+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366544+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357002+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370005+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.367198+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.370857+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:21:46.205676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.365523+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:34:49.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366399+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373506+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.364080+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373181+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369153+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373285+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-08-15T16:59:39.475362+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:39.475362+0000","last_peered":"2024-08-15T16:59:39.475362+0000","last_clean":"2024-08-15T16:59:39.475362+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:39.475362+0000","last_undegraded":"2024-08-15T16:59:39.475362+0000","last_fullsized":"2024-08-15T16:59:39.475362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357832+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.373221+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.356721+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369228+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.359423+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369653+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.366021+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.358937+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.369167+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:17:38.398837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.368697+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:03:50.710123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357835+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.357941+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.360200+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.363800+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:25:15.493688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-15T16:59:41.349871+0000","last_change":"2024-08-15T16:59:41.349871+0000","last_active":"2024-08-15T16:59:41.349871+0000","last_peered":"2024-08-15T16:59:41.349871+0000","last_clean":"2024-08-15T16:59:41.349871+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:41.349871+0000","last_undegraded":"2024-08-15T16:59:41.349871+0000","last_fullsized":"2024-08-15T16:59:41.349871+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-15T16:59:40.339920+0000","last_change":"2024-08-15T16:59:40.371247+0000","last_active":"2024-08-15T16:59:40.339920+0000","last_peered":"2024-08-15T16:59:40.339920+0000","last_clean":"2024-08-15T16:59:40.339920+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-15T16:59:40.339920+0000","last_undegraded":"2024-08-15T16:59:40.339920+0000","last_fullsized":"2024-08-15T16:59:40.339920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:44:42.440651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298220,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073620,"statfs":{"total":96636764160,"available":96331386880,"internally_reserved":0,"allocated":757760,"data_stored":639038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542146,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297516,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074324,"statfs":{"total":96636764160,"available":96332107776,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4259999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.843}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":176128,"data_stored":54475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":163840,"data_stored":48670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2230000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.96}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7190000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":753664,"data_stored":638077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5860000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:46.849 INFO:tasks.ceph.mds.b.smithi161.stderr:2024-08-15T16:59:46.837+0000 7ff6331fe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T16:59:46.995 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T16:59:46.998 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T16:59:49.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:49.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 Processor -- start 2024-08-15T16:59:49.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 -- start start 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf400671d0 0x7fcf400b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 0x7fcf400686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf40150040 con 0x7fcf40066e00 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf400af730 con 0x7fcf400671d0 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf44d83700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf40155b30 con 0x7fcf400b21b0 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf400671d0 0x7fcf400b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf400671d0 0x7fcf400b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49580/0 (socket says 172.21.15.146:49580) 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 -- 172.21.15.146:0/3952612218 learned_addr learned my addr 172.21.15.146:0/3952612218 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3f7fe700 1 --1- 172.21.15.146:0/3952612218 >> v1:172.21.15.146:6789/0 conn(0x7fcf40066e00 0x7fcf4014f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60042/0 (socket says 172.21.15.146:60042) 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3ffff700 1 --2- 172.21.15.146:0/3952612218 >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 0x7fcf400686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 -- 172.21.15.146:0/3952612218 >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 msgr2=0x7fcf400686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 --2- 172.21.15.146:0/3952612218 >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 0x7fcf400686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 -- 172.21.15.146:0/3952612218 >> v1:172.21.15.146:6789/0 conn(0x7fcf40066e00 legacy=0x7fcf4014f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 -- 172.21.15.146:0/3952612218 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf40155df0 con 0x7fcf400671d0 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3ffff700 1 --2- 172.21.15.146:0/3952612218 >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 0x7fcf400686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3effd700 1 --2- 172.21.15.146:0/3952612218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf400b1c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf40153700 tx=0x7fcf38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129a2fadc6f82708 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3e7fc700 1 -- 172.21.15.146:0/3952612218 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3800e070 con 0x7fcf400671d0 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3e7fc700 1 -- 172.21.15.146:0/3952612218 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3800ba50 con 0x7fcf400671d0 2024-08-15T16:59:49.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.877+0000 7fcf3e7fc700 1 -- 172.21.15.146:0/3952612218 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf380092e0 con 0x7fcf400671d0 2024-08-15T16:59:49.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- 172.21.15.146:0/3952612218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 msgr2=0x7fcf400b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:49.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 --2- 172.21.15.146:0/3952612218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf400b1c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf40153700 tx=0x7fcf38009840 comp rx=0 tx=0).stop 2024-08-15T16:59:49.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- 172.21.15.146:0/3952612218 shutdown_connections 2024-08-15T16:59:49.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 --2- 172.21.15.146:0/3952612218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf400b1c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:49.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 --2- 172.21.15.146:0/3952612218 >> v2:172.21.15.161:3300/0 conn(0x7fcf400b21b0 0x7fcf400686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:49.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- 172.21.15.146:0/3952612218 >> 172.21.15.146:0/3952612218 conn(0x7fcf4006d790 msgr2=0x7fcf4006db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:49.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- 172.21.15.146:0/3952612218 shutdown_connections 2024-08-15T16:59:49.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- 172.21.15.146:0/3952612218 wait complete. 2024-08-15T16:59:49.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 Processor -- start 2024-08-15T16:59:49.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.881+0000 7fcf44d83700 1 -- start start 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 0x7fcf40148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf400b21b0 0x7fcf4013cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40145f60 con 0x7fcf400671d0 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40152550 con 0x7fcf40066e00 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401495b0 con 0x7fcf400b21b0 2024-08-15T16:59:49.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49596/0 (socket says 172.21.15.146:49596) 2024-08-15T16:59:49.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 -- 172.21.15.146:0/2330035130 learned_addr learned my addr 172.21.15.146:0/2330035130 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:49.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3f7fe700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 0x7fcf40148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3ffff700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf400b21b0 0x7fcf4013cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 -- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf400b21b0 msgr2=0x7fcf4013cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf400b21b0 0x7fcf4013cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 -- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 msgr2=0x7fcf40148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 0x7fcf40148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 -- 172.21.15.146:0/2330035130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf38009ae0 con 0x7fcf400671d0 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3f7fe700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 0x7fcf40148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf3effd700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006efb0 tx=0x7fcf38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38008aa0 con 0x7fcf400671d0 2024-08-15T16:59:49.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf38008c40 con 0x7fcf400671d0 2024-08-15T16:59:49.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf38026460 con 0x7fcf400671d0 2024-08-15T16:59:49.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf40149a40 con 0x7fcf400671d0 2024-08-15T16:59:49.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401489a0 con 0x7fcf400671d0 2024-08-15T16:59:49.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.885+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400643a0 con 0x7fcf400671d0 2024-08-15T16:59:49.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.889+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcf38003040 con 0x7fcf400671d0 2024-08-15T16:59:49.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.889+0000 7fcf377fe700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 0x7fcf20062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:49.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.889+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf3807eac0 con 0x7fcf400671d0 2024-08-15T16:59:49.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.889+0000 7fcf3f7fe700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 0x7fcf20062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:49.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.889+0000 7fcf3f7fe700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 0x7fcf20062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28007d40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:49.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:49.897+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcf3804ef10 con 0x7fcf400671d0 2024-08-15T16:59:50.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.061+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf400ad130 con 0x7fcf20060860 2024-08-15T16:59:50.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.077+0000 7fcf377fe700 1 -- 172.21.15.146:0/2330035130 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264551 (secure 0 0 0) 0x7fcf400ad130 con 0x7fcf20060860 2024-08-15T16:59:50.094 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 msgr2=0x7fcf20062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 0x7fcf20062d10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28007d40 comp rx=0 tx=0).stop 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 msgr2=0x7fcf40152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006efb0 tx=0x7fcf38018000 comp rx=0 tx=0).stop 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 shutdown_connections 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcf20060860 0x7fcf20062d10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf400b21b0 0x7fcf4013cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf400671d0 0x7fcf40152010 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 --2- 172.21.15.146:0/2330035130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf40066e00 0x7fcf40148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 >> 172.21.15.146:0/2330035130 conn(0x7fcf4006d790 msgr2=0x7fcf4014f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 shutdown_connections 2024-08-15T16:59:50.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.085+0000 7fcf44d83700 1 -- 172.21.15.146:0/2330035130 wait complete. 2024-08-15T16:59:50.097 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:50.109 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-08-15T16:59:49.645823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":121,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385912,"kb_used_data":2884,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588808,"statfs":{"total":773094113280,"available":770650939392,"internally_reserved":0,"allocated":2953216,"data_stored":1774763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002923"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413334+0000","last_change":"2024-08-15T16:59:42.409245+0000","last_active":"2024-08-15T16:59:42.413334+0000","last_peered":"2024-08-15T16:59:42.413334+0000","last_clean":"2024-08-15T16:59:42.413334+0000","last_became_active":"2024-08-15T16:59:42.408987+0000","last_became_peered":"2024-08-15T16:59:42.408987+0000","last_unstale":"2024-08-15T16:59:42.413334+0000","last_undegraded":"2024-08-15T16:59:42.413334+0000","last_fullsized":"2024-08-15T16:59:42.413334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:01:07.089558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422299+0000","last_change":"2024-08-15T16:59:42.418957+0000","last_active":"2024-08-15T16:59:42.422299+0000","last_peered":"2024-08-15T16:59:42.422299+0000","last_clean":"2024-08-15T16:59:42.422299+0000","last_became_active":"2024-08-15T16:59:42.418575+0000","last_became_peered":"2024-08-15T16:59:42.418575+0000","last_unstale":"2024-08-15T16:59:42.422299+0000","last_undegraded":"2024-08-15T16:59:42.422299+0000","last_fullsized":"2024-08-15T16:59:42.422299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419909+0000","last_change":"2024-08-15T16:59:42.410104+0000","last_active":"2024-08-15T16:59:42.419909+0000","last_peered":"2024-08-15T16:59:42.419909+0000","last_clean":"2024-08-15T16:59:42.419909+0000","last_became_active":"2024-08-15T16:59:42.409813+0000","last_became_peered":"2024-08-15T16:59:42.409813+0000","last_unstale":"2024-08-15T16:59:42.419909+0000","last_undegraded":"2024-08-15T16:59:42.419909+0000","last_fullsized":"2024-08-15T16:59:42.419909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:00.063666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416866+0000","last_change":"2024-08-15T16:59:42.415914+0000","last_active":"2024-08-15T16:59:42.416866+0000","last_peered":"2024-08-15T16:59:42.416866+0000","last_clean":"2024-08-15T16:59:42.416866+0000","last_became_active":"2024-08-15T16:59:42.415646+0000","last_became_peered":"2024-08-15T16:59:42.415646+0000","last_unstale":"2024-08-15T16:59:42.416866+0000","last_undegraded":"2024-08-15T16:59:42.416866+0000","last_fullsized":"2024-08-15T16:59:42.416866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:10:45.984041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413099+0000","last_change":"2024-08-15T16:59:42.406837+0000","last_active":"2024-08-15T16:59:42.413099+0000","last_peered":"2024-08-15T16:59:42.413099+0000","last_clean":"2024-08-15T16:59:42.413099+0000","last_became_active":"2024-08-15T16:59:42.406562+0000","last_became_peered":"2024-08-15T16:59:42.406562+0000","last_unstale":"2024-08-15T16:59:42.413099+0000","last_undegraded":"2024-08-15T16:59:42.413099+0000","last_fullsized":"2024-08-15T16:59:42.413099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:07:47.921825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414600+0000","last_change":"2024-08-15T16:59:42.403625+0000","last_active":"2024-08-15T16:59:42.414600+0000","last_peered":"2024-08-15T16:59:42.414600+0000","last_clean":"2024-08-15T16:59:42.414600+0000","last_became_active":"2024-08-15T16:59:42.403339+0000","last_became_peered":"2024-08-15T16:59:42.403339+0000","last_unstale":"2024-08-15T16:59:42.414600+0000","last_undegraded":"2024-08-15T16:59:42.414600+0000","last_fullsized":"2024-08-15T16:59:42.414600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:30.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417227+0000","last_change":"2024-08-15T16:59:42.413170+0000","last_active":"2024-08-15T16:59:42.417227+0000","last_peered":"2024-08-15T16:59:42.417227+0000","last_clean":"2024-08-15T16:59:42.417227+0000","last_became_active":"2024-08-15T16:59:42.409523+0000","last_became_peered":"2024-08-15T16:59:42.409523+0000","last_unstale":"2024-08-15T16:59:42.417227+0000","last_undegraded":"2024-08-15T16:59:42.417227+0000","last_fullsized":"2024-08-15T16:59:42.417227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:25.911685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418927+0000","last_change":"2024-08-15T16:59:42.412418+0000","last_active":"2024-08-15T16:59:42.418927+0000","last_peered":"2024-08-15T16:59:42.418927+0000","last_clean":"2024-08-15T16:59:42.418927+0000","last_became_active":"2024-08-15T16:59:42.412174+0000","last_became_peered":"2024-08-15T16:59:42.412174+0000","last_unstale":"2024-08-15T16:59:42.418927+0000","last_undegraded":"2024-08-15T16:59:42.418927+0000","last_fullsized":"2024-08-15T16:59:42.418927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:00:19.102852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419430+0000","last_change":"2024-08-15T16:59:42.416969+0000","last_active":"2024-08-15T16:59:42.419430+0000","last_peered":"2024-08-15T16:59:42.419430+0000","last_clean":"2024-08-15T16:59:42.419430+0000","last_became_active":"2024-08-15T16:59:42.416665+0000","last_became_peered":"2024-08-15T16:59:42.416665+0000","last_unstale":"2024-08-15T16:59:42.419430+0000","last_undegraded":"2024-08-15T16:59:42.419430+0000","last_fullsized":"2024-08-15T16:59:42.419430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:31:42.855850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401340+0000","last_change":"2024-08-15T16:59:42.399470+0000","last_active":"2024-08-15T16:59:42.401340+0000","last_peered":"2024-08-15T16:59:42.401340+0000","last_clean":"2024-08-15T16:59:42.401340+0000","last_became_active":"2024-08-15T16:59:42.398232+0000","last_became_peered":"2024-08-15T16:59:42.398232+0000","last_unstale":"2024-08-15T16:59:42.401340+0000","last_undegraded":"2024-08-15T16:59:42.401340+0000","last_fullsized":"2024-08-15T16:59:42.401340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401337+0000","last_change":"2024-08-15T16:59:42.399759+0000","last_active":"2024-08-15T16:59:42.401337+0000","last_peered":"2024-08-15T16:59:42.401337+0000","last_clean":"2024-08-15T16:59:42.401337+0000","last_became_active":"2024-08-15T16:59:42.398288+0000","last_became_peered":"2024-08-15T16:59:42.398288+0000","last_unstale":"2024-08-15T16:59:42.401337+0000","last_undegraded":"2024-08-15T16:59:42.401337+0000","last_fullsized":"2024-08-15T16:59:42.401337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.402316+0000","last_change":"2024-08-15T16:59:42.397942+0000","last_active":"2024-08-15T16:59:42.402316+0000","last_peered":"2024-08-15T16:59:42.402316+0000","last_clean":"2024-08-15T16:59:42.402316+0000","last_became_active":"2024-08-15T16:59:42.397617+0000","last_became_peered":"2024-08-15T16:59:42.397617+0000","last_unstale":"2024-08-15T16:59:42.402316+0000","last_undegraded":"2024-08-15T16:59:42.402316+0000","last_fullsized":"2024-08-15T16:59:42.402316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:48:39.070398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416917+0000","last_change":"2024-08-15T16:59:42.408020+0000","last_active":"2024-08-15T16:59:42.416917+0000","last_peered":"2024-08-15T16:59:42.416917+0000","last_clean":"2024-08-15T16:59:42.416917+0000","last_became_active":"2024-08-15T16:59:42.407730+0000","last_became_peered":"2024-08-15T16:59:42.407730+0000","last_unstale":"2024-08-15T16:59:42.416917+0000","last_undegraded":"2024-08-15T16:59:42.416917+0000","last_fullsized":"2024-08-15T16:59:42.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420317+0000","last_change":"2024-08-15T16:59:42.415216+0000","last_active":"2024-08-15T16:59:42.420317+0000","last_peered":"2024-08-15T16:59:42.420317+0000","last_clean":"2024-08-15T16:59:42.420317+0000","last_became_active":"2024-08-15T16:59:42.414950+0000","last_became_peered":"2024-08-15T16:59:42.414950+0000","last_unstale":"2024-08-15T16:59:42.420317+0000","last_undegraded":"2024-08-15T16:59:42.420317+0000","last_fullsized":"2024-08-15T16:59:42.420317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:47:13.915180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411187+0000","last_change":"2024-08-15T16:59:42.403579+0000","last_active":"2024-08-15T16:59:42.411187+0000","last_peered":"2024-08-15T16:59:42.411187+0000","last_clean":"2024-08-15T16:59:42.411187+0000","last_became_active":"2024-08-15T16:59:42.403244+0000","last_became_peered":"2024-08-15T16:59:42.403244+0000","last_unstale":"2024-08-15T16:59:42.411187+0000","last_undegraded":"2024-08-15T16:59:42.411187+0000","last_fullsized":"2024-08-15T16:59:42.411187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414413+0000","last_change":"2024-08-15T16:59:42.407583+0000","last_active":"2024-08-15T16:59:42.414413+0000","last_peered":"2024-08-15T16:59:42.414413+0000","last_clean":"2024-08-15T16:59:42.414413+0000","last_became_active":"2024-08-15T16:59:42.406689+0000","last_became_peered":"2024-08-15T16:59:42.406689+0000","last_unstale":"2024-08-15T16:59:42.414413+0000","last_undegraded":"2024-08-15T16:59:42.414413+0000","last_fullsized":"2024-08-15T16:59:42.414413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:49:48.769242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414426+0000","last_change":"2024-08-15T16:59:42.404508+0000","last_active":"2024-08-15T16:59:42.414426+0000","last_peered":"2024-08-15T16:59:42.414426+0000","last_clean":"2024-08-15T16:59:42.414426+0000","last_became_active":"2024-08-15T16:59:42.404223+0000","last_became_peered":"2024-08-15T16:59:42.404223+0000","last_unstale":"2024-08-15T16:59:42.414426+0000","last_undegraded":"2024-08-15T16:59:42.414426+0000","last_fullsized":"2024-08-15T16:59:42.414426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:07:35.144380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.403776+0000","last_change":"2024-08-15T16:59:42.400163+0000","last_active":"2024-08-15T16:59:42.403776+0000","last_peered":"2024-08-15T16:59:42.403776+0000","last_clean":"2024-08-15T16:59:42.403776+0000","last_became_active":"2024-08-15T16:59:42.399738+0000","last_became_peered":"2024-08-15T16:59:42.399738+0000","last_unstale":"2024-08-15T16:59:42.403776+0000","last_undegraded":"2024-08-15T16:59:42.403776+0000","last_fullsized":"2024-08-15T16:59:42.403776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:40.046084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413534+0000","last_change":"2024-08-15T16:59:42.403318+0000","last_active":"2024-08-15T16:59:42.413534+0000","last_peered":"2024-08-15T16:59:42.413534+0000","last_clean":"2024-08-15T16:59:42.413534+0000","last_became_active":"2024-08-15T16:59:42.403051+0000","last_became_peered":"2024-08-15T16:59:42.403051+0000","last_unstale":"2024-08-15T16:59:42.413534+0000","last_undegraded":"2024-08-15T16:59:42.413534+0000","last_fullsized":"2024-08-15T16:59:42.413534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401457+0000","last_change":"2024-08-15T16:59:42.397107+0000","last_active":"2024-08-15T16:59:42.401457+0000","last_peered":"2024-08-15T16:59:42.401457+0000","last_clean":"2024-08-15T16:59:42.401457+0000","last_became_active":"2024-08-15T16:59:42.396786+0000","last_became_peered":"2024-08-15T16:59:42.396786+0000","last_unstale":"2024-08-15T16:59:42.401457+0000","last_undegraded":"2024-08-15T16:59:42.401457+0000","last_fullsized":"2024-08-15T16:59:42.401457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422555+0000","last_change":"2024-08-15T16:59:42.418246+0000","last_active":"2024-08-15T16:59:42.422555+0000","last_peered":"2024-08-15T16:59:42.422555+0000","last_clean":"2024-08-15T16:59:42.422555+0000","last_became_active":"2024-08-15T16:59:42.417975+0000","last_became_peered":"2024-08-15T16:59:42.417975+0000","last_unstale":"2024-08-15T16:59:42.422555+0000","last_undegraded":"2024-08-15T16:59:42.422555+0000","last_fullsized":"2024-08-15T16:59:42.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:46:29.993181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414618+0000","last_change":"2024-08-15T16:59:42.405054+0000","last_active":"2024-08-15T16:59:42.414618+0000","last_peered":"2024-08-15T16:59:42.414618+0000","last_clean":"2024-08-15T16:59:42.414618+0000","last_became_active":"2024-08-15T16:59:42.404788+0000","last_became_peered":"2024-08-15T16:59:42.404788+0000","last_unstale":"2024-08-15T16:59:42.414618+0000","last_undegraded":"2024-08-15T16:59:42.414618+0000","last_fullsized":"2024-08-15T16:59:42.414618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:49:25.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417014+0000","last_change":"2024-08-15T16:59:42.408237+0000","last_active":"2024-08-15T16:59:42.417014+0000","last_peered":"2024-08-15T16:59:42.417014+0000","last_clean":"2024-08-15T16:59:42.417014+0000","last_became_active":"2024-08-15T16:59:42.407321+0000","last_became_peered":"2024-08-15T16:59:42.407321+0000","last_unstale":"2024-08-15T16:59:42.417014+0000","last_undegraded":"2024-08-15T16:59:42.417014+0000","last_fullsized":"2024-08-15T16:59:42.417014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414394+0000","last_change":"2024-08-15T16:59:42.406550+0000","last_active":"2024-08-15T16:59:42.414394+0000","last_peered":"2024-08-15T16:59:42.414394+0000","last_clean":"2024-08-15T16:59:42.414394+0000","last_became_active":"2024-08-15T16:59:42.406280+0000","last_became_peered":"2024-08-15T16:59:42.406280+0000","last_unstale":"2024-08-15T16:59:42.414394+0000","last_undegraded":"2024-08-15T16:59:42.414394+0000","last_fullsized":"2024-08-15T16:59:42.414394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:47:40.336658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416946+0000","last_change":"2024-08-15T16:59:42.408293+0000","last_active":"2024-08-15T16:59:42.416946+0000","last_peered":"2024-08-15T16:59:42.416946+0000","last_clean":"2024-08-15T16:59:42.416946+0000","last_became_active":"2024-08-15T16:59:42.408005+0000","last_became_peered":"2024-08-15T16:59:42.408005+0000","last_unstale":"2024-08-15T16:59:42.416946+0000","last_undegraded":"2024-08-15T16:59:42.416946+0000","last_fullsized":"2024-08-15T16:59:42.416946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.408666+0000","last_change":"2024-08-15T16:59:42.403047+0000","last_active":"2024-08-15T16:59:42.408666+0000","last_peered":"2024-08-15T16:59:42.408666+0000","last_clean":"2024-08-15T16:59:42.408666+0000","last_became_active":"2024-08-15T16:59:42.402801+0000","last_became_peered":"2024-08-15T16:59:42.402801+0000","last_unstale":"2024-08-15T16:59:42.408666+0000","last_undegraded":"2024-08-15T16:59:42.408666+0000","last_fullsized":"2024-08-15T16:59:42.408666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416956+0000","last_change":"2024-08-15T16:59:42.411339+0000","last_active":"2024-08-15T16:59:42.416956+0000","last_peered":"2024-08-15T16:59:42.416956+0000","last_clean":"2024-08-15T16:59:42.416956+0000","last_became_active":"2024-08-15T16:59:42.411055+0000","last_became_peered":"2024-08-15T16:59:42.411055+0000","last_unstale":"2024-08-15T16:59:42.416956+0000","last_undegraded":"2024-08-15T16:59:42.416956+0000","last_fullsized":"2024-08-15T16:59:42.416956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417334+0000","last_change":"2024-08-15T16:59:42.411492+0000","last_active":"2024-08-15T16:59:42.417334+0000","last_peered":"2024-08-15T16:59:42.417334+0000","last_clean":"2024-08-15T16:59:42.417334+0000","last_became_active":"2024-08-15T16:59:42.411182+0000","last_became_peered":"2024-08-15T16:59:42.411182+0000","last_unstale":"2024-08-15T16:59:42.417334+0000","last_undegraded":"2024-08-15T16:59:42.417334+0000","last_fullsized":"2024-08-15T16:59:42.417334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420116+0000","last_change":"2024-08-15T16:59:42.412347+0000","last_active":"2024-08-15T16:59:42.420116+0000","last_peered":"2024-08-15T16:59:42.420116+0000","last_clean":"2024-08-15T16:59:42.420116+0000","last_became_active":"2024-08-15T16:59:42.411964+0000","last_became_peered":"2024-08-15T16:59:42.411964+0000","last_unstale":"2024-08-15T16:59:42.420116+0000","last_undegraded":"2024-08-15T16:59:42.420116+0000","last_fullsized":"2024-08-15T16:59:42.420116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:21.481110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414638+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.414638+0000","last_peered":"2024-08-15T16:59:42.414638+0000","last_clean":"2024-08-15T16:59:42.414638+0000","last_became_active":"2024-08-15T16:59:42.407063+0000","last_became_peered":"2024-08-15T16:59:42.407063+0000","last_unstale":"2024-08-15T16:59:42.414638+0000","last_undegraded":"2024-08-15T16:59:42.414638+0000","last_fullsized":"2024-08-15T16:59:42.414638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:54:52.191383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.405736+0000","last_change":"2024-08-15T16:59:42.398430+0000","last_active":"2024-08-15T16:59:42.405736+0000","last_peered":"2024-08-15T16:59:42.405736+0000","last_clean":"2024-08-15T16:59:42.405736+0000","last_became_active":"2024-08-15T16:59:42.397995+0000","last_became_peered":"2024-08-15T16:59:42.397995+0000","last_unstale":"2024-08-15T16:59:42.405736+0000","last_undegraded":"2024-08-15T16:59:42.405736+0000","last_fullsized":"2024-08-15T16:59:42.405736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:02.439971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418173+0000","last_change":"2024-08-15T16:59:42.417359+0000","last_active":"2024-08-15T16:59:42.418173+0000","last_peered":"2024-08-15T16:59:42.418173+0000","last_clean":"2024-08-15T16:59:42.418173+0000","last_became_active":"2024-08-15T16:59:42.416784+0000","last_became_peered":"2024-08-15T16:59:42.416784+0000","last_unstale":"2024-08-15T16:59:42.418173+0000","last_undegraded":"2024-08-15T16:59:42.418173+0000","last_fullsized":"2024-08-15T16:59:42.418173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400566+0000","last_change":"2024-08-15T16:59:42.396288+0000","last_active":"2024-08-15T16:59:42.400566+0000","last_peered":"2024-08-15T16:59:42.400566+0000","last_clean":"2024-08-15T16:59:42.400566+0000","last_became_active":"2024-08-15T16:59:42.395944+0000","last_became_peered":"2024-08-15T16:59:42.395944+0000","last_unstale":"2024-08-15T16:59:42.400566+0000","last_undegraded":"2024-08-15T16:59:42.400566+0000","last_fullsized":"2024-08-15T16:59:42.400566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419597+0000","last_change":"2024-08-15T16:59:41.381424+0000","last_active":"2024-08-15T16:59:42.419597+0000","last_peered":"2024-08-15T16:59:42.419597+0000","last_clean":"2024-08-15T16:59:42.419597+0000","last_became_active":"2024-08-15T16:59:41.381154+0000","last_became_peered":"2024-08-15T16:59:41.381154+0000","last_unstale":"2024-08-15T16:59:42.419597+0000","last_undegraded":"2024-08-15T16:59:42.419597+0000","last_fullsized":"2024-08-15T16:59:42.419597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420098+0000","last_change":"2024-08-15T16:59:42.412224+0000","last_active":"2024-08-15T16:59:42.420098+0000","last_peered":"2024-08-15T16:59:42.420098+0000","last_clean":"2024-08-15T16:59:42.420098+0000","last_became_active":"2024-08-15T16:59:42.411384+0000","last_became_peered":"2024-08-15T16:59:42.411384+0000","last_unstale":"2024-08-15T16:59:42.420098+0000","last_undegraded":"2024-08-15T16:59:42.420098+0000","last_fullsized":"2024-08-15T16:59:42.420098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:28:38.772967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.415389+0000","last_change":"2024-08-15T16:59:41.370657+0000","last_active":"2024-08-15T16:59:42.415389+0000","last_peered":"2024-08-15T16:59:42.415389+0000","last_clean":"2024-08-15T16:59:42.415389+0000","last_became_active":"2024-08-15T16:59:41.370303+0000","last_became_peered":"2024-08-15T16:59:41.370303+0000","last_unstale":"2024-08-15T16:59:42.415389+0000","last_undegraded":"2024-08-15T16:59:42.415389+0000","last_fullsized":"2024-08-15T16:59:42.415389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398754+0000","last_change":"2024-08-15T16:59:41.376440+0000","last_active":"2024-08-15T16:59:42.398754+0000","last_peered":"2024-08-15T16:59:42.398754+0000","last_clean":"2024-08-15T16:59:42.398754+0000","last_became_active":"2024-08-15T16:59:41.375356+0000","last_became_peered":"2024-08-15T16:59:41.375356+0000","last_unstale":"2024-08-15T16:59:42.398754+0000","last_undegraded":"2024-08-15T16:59:42.398754+0000","last_fullsized":"2024-08-15T16:59:42.398754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418845+0000","last_change":"2024-08-15T16:59:42.409055+0000","last_active":"2024-08-15T16:59:42.418845+0000","last_peered":"2024-08-15T16:59:42.418845+0000","last_clean":"2024-08-15T16:59:42.418845+0000","last_became_active":"2024-08-15T16:59:42.408704+0000","last_became_peered":"2024-08-15T16:59:42.408704+0000","last_unstale":"2024-08-15T16:59:42.418845+0000","last_undegraded":"2024-08-15T16:59:42.418845+0000","last_fullsized":"2024-08-15T16:59:42.418845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.388705+0000","last_change":"2024-08-15T16:59:41.372714+0000","last_active":"2024-08-15T16:59:42.388705+0000","last_peered":"2024-08-15T16:59:42.388705+0000","last_clean":"2024-08-15T16:59:42.388705+0000","last_became_active":"2024-08-15T16:59:41.372369+0000","last_became_peered":"2024-08-15T16:59:41.372369+0000","last_unstale":"2024-08-15T16:59:42.388705+0000","last_undegraded":"2024-08-15T16:59:42.388705+0000","last_fullsized":"2024-08-15T16:59:42.388705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:18.195005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417279+0000","last_change":"2024-08-15T16:59:42.412973+0000","last_active":"2024-08-15T16:59:42.417279+0000","last_peered":"2024-08-15T16:59:42.417279+0000","last_clean":"2024-08-15T16:59:42.417279+0000","last_became_active":"2024-08-15T16:59:42.412668+0000","last_became_peered":"2024-08-15T16:59:42.412668+0000","last_unstale":"2024-08-15T16:59:42.417279+0000","last_undegraded":"2024-08-15T16:59:42.417279+0000","last_fullsized":"2024-08-15T16:59:42.417279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.394506+0000","last_change":"2024-08-15T16:59:41.371198+0000","last_active":"2024-08-15T16:59:42.394506+0000","last_peered":"2024-08-15T16:59:42.394506+0000","last_clean":"2024-08-15T16:59:42.394506+0000","last_became_active":"2024-08-15T16:59:41.370260+0000","last_became_peered":"2024-08-15T16:59:41.370260+0000","last_unstale":"2024-08-15T16:59:42.394506+0000","last_undegraded":"2024-08-15T16:59:42.394506+0000","last_fullsized":"2024-08-15T16:59:42.394506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414362+0000","last_change":"2024-08-15T16:59:42.403938+0000","last_active":"2024-08-15T16:59:42.414362+0000","last_peered":"2024-08-15T16:59:42.414362+0000","last_clean":"2024-08-15T16:59:42.414362+0000","last_became_active":"2024-08-15T16:59:42.403693+0000","last_became_peered":"2024-08-15T16:59:42.403693+0000","last_unstale":"2024-08-15T16:59:42.414362+0000","last_undegraded":"2024-08-15T16:59:42.414362+0000","last_fullsized":"2024-08-15T16:59:42.414362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:46.202413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.390318+0000","last_change":"2024-08-15T16:59:41.370700+0000","last_active":"2024-08-15T16:59:42.390318+0000","last_peered":"2024-08-15T16:59:42.390318+0000","last_clean":"2024-08-15T16:59:42.390318+0000","last_became_active":"2024-08-15T16:59:41.370333+0000","last_became_peered":"2024-08-15T16:59:41.370333+0000","last_unstale":"2024-08-15T16:59:42.390318+0000","last_undegraded":"2024-08-15T16:59:42.390318+0000","last_fullsized":"2024-08-15T16:59:42.390318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:40.170510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420410+0000","last_change":"2024-08-15T16:59:42.415577+0000","last_active":"2024-08-15T16:59:42.420410+0000","last_peered":"2024-08-15T16:59:42.420410+0000","last_clean":"2024-08-15T16:59:42.420410+0000","last_became_active":"2024-08-15T16:59:42.415259+0000","last_became_peered":"2024-08-15T16:59:42.415259+0000","last_unstale":"2024-08-15T16:59:42.420410+0000","last_undegraded":"2024-08-15T16:59:42.420410+0000","last_fullsized":"2024-08-15T16:59:42.420410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:31.974939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420880+0000","last_change":"2024-08-15T16:59:41.382088+0000","last_active":"2024-08-15T16:59:42.420880+0000","last_peered":"2024-08-15T16:59:42.420880+0000","last_clean":"2024-08-15T16:59:42.420880+0000","last_became_active":"2024-08-15T16:59:41.380881+0000","last_became_peered":"2024-08-15T16:59:41.380881+0000","last_unstale":"2024-08-15T16:59:42.420880+0000","last_undegraded":"2024-08-15T16:59:42.420880+0000","last_fullsized":"2024-08-15T16:59:42.420880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.404994+0000","last_change":"2024-08-15T16:59:42.403887+0000","last_active":"2024-08-15T16:59:42.404994+0000","last_peered":"2024-08-15T16:59:42.404994+0000","last_clean":"2024-08-15T16:59:42.404994+0000","last_became_active":"2024-08-15T16:59:42.403619+0000","last_became_peered":"2024-08-15T16:59:42.403619+0000","last_unstale":"2024-08-15T16:59:42.404994+0000","last_undegraded":"2024-08-15T16:59:42.404994+0000","last_fullsized":"2024-08-15T16:59:42.404994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421139+0000","last_change":"2024-08-15T16:59:41.379757+0000","last_active":"2024-08-15T16:59:42.421139+0000","last_peered":"2024-08-15T16:59:42.421139+0000","last_clean":"2024-08-15T16:59:42.421139+0000","last_became_active":"2024-08-15T16:59:41.379393+0000","last_became_peered":"2024-08-15T16:59:41.379393+0000","last_unstale":"2024-08-15T16:59:42.421139+0000","last_undegraded":"2024-08-15T16:59:42.421139+0000","last_fullsized":"2024-08-15T16:59:42.421139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418889+0000","last_change":"2024-08-15T16:59:42.413445+0000","last_active":"2024-08-15T16:59:42.418889+0000","last_peered":"2024-08-15T16:59:42.418889+0000","last_clean":"2024-08-15T16:59:42.418889+0000","last_became_active":"2024-08-15T16:59:42.413176+0000","last_became_peered":"2024-08-15T16:59:42.413176+0000","last_unstale":"2024-08-15T16:59:42.418889+0000","last_undegraded":"2024-08-15T16:59:42.418889+0000","last_fullsized":"2024-08-15T16:59:42.418889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420263+0000","last_change":"2024-08-15T16:59:41.381453+0000","last_active":"2024-08-15T16:59:42.420263+0000","last_peered":"2024-08-15T16:59:42.420263+0000","last_clean":"2024-08-15T16:59:42.420263+0000","last_became_active":"2024-08-15T16:59:41.381090+0000","last_became_peered":"2024-08-15T16:59:41.381090+0000","last_unstale":"2024-08-15T16:59:42.420263+0000","last_undegraded":"2024-08-15T16:59:42.420263+0000","last_fullsized":"2024-08-15T16:59:42.420263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420060+0000","last_change":"2024-08-15T16:59:42.411131+0000","last_active":"2024-08-15T16:59:42.420060+0000","last_peered":"2024-08-15T16:59:42.420060+0000","last_clean":"2024-08-15T16:59:42.420060+0000","last_became_active":"2024-08-15T16:59:42.410756+0000","last_became_peered":"2024-08-15T16:59:42.410756+0000","last_unstale":"2024-08-15T16:59:42.420060+0000","last_undegraded":"2024-08-15T16:59:42.420060+0000","last_fullsized":"2024-08-15T16:59:42.420060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391610+0000","last_change":"2024-08-15T16:59:41.373319+0000","last_active":"2024-08-15T16:59:42.391610+0000","last_peered":"2024-08-15T16:59:42.391610+0000","last_clean":"2024-08-15T16:59:42.391610+0000","last_became_active":"2024-08-15T16:59:41.369550+0000","last_became_peered":"2024-08-15T16:59:41.369550+0000","last_unstale":"2024-08-15T16:59:42.391610+0000","last_undegraded":"2024-08-15T16:59:42.391610+0000","last_fullsized":"2024-08-15T16:59:42.391610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:16:34.424937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413118+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.413118+0000","last_peered":"2024-08-15T16:59:42.413118+0000","last_clean":"2024-08-15T16:59:42.413118+0000","last_became_active":"2024-08-15T16:59:42.407033+0000","last_became_peered":"2024-08-15T16:59:42.407033+0000","last_unstale":"2024-08-15T16:59:42.413118+0000","last_undegraded":"2024-08-15T16:59:42.413118+0000","last_fullsized":"2024-08-15T16:59:42.413118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:58.883271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.392955+0000","last_change":"2024-08-15T16:59:41.370530+0000","last_active":"2024-08-15T16:59:42.392955+0000","last_peered":"2024-08-15T16:59:42.392955+0000","last_clean":"2024-08-15T16:59:42.392955+0000","last_became_active":"2024-08-15T16:59:41.369807+0000","last_became_peered":"2024-08-15T16:59:41.369807+0000","last_unstale":"2024-08-15T16:59:42.392955+0000","last_undegraded":"2024-08-15T16:59:42.392955+0000","last_fullsized":"2024-08-15T16:59:42.392955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418194+0000","last_change":"2024-08-15T16:59:42.417158+0000","last_active":"2024-08-15T16:59:42.418194+0000","last_peered":"2024-08-15T16:59:42.418194+0000","last_clean":"2024-08-15T16:59:42.418194+0000","last_became_active":"2024-08-15T16:59:42.416838+0000","last_became_peered":"2024-08-15T16:59:42.416838+0000","last_unstale":"2024-08-15T16:59:42.418194+0000","last_undegraded":"2024-08-15T16:59:42.418194+0000","last_fullsized":"2024-08-15T16:59:42.418194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422895+0000","last_change":"2024-08-15T16:59:41.377962+0000","last_active":"2024-08-15T16:59:42.422895+0000","last_peered":"2024-08-15T16:59:42.422895+0000","last_clean":"2024-08-15T16:59:42.422895+0000","last_became_active":"2024-08-15T16:59:41.377699+0000","last_became_peered":"2024-08-15T16:59:41.377699+0000","last_unstale":"2024-08-15T16:59:42.422895+0000","last_undegraded":"2024-08-15T16:59:42.422895+0000","last_fullsized":"2024-08-15T16:59:42.422895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413284+0000","last_change":"2024-08-15T16:59:42.407316+0000","last_active":"2024-08-15T16:59:42.413284+0000","last_peered":"2024-08-15T16:59:42.413284+0000","last_clean":"2024-08-15T16:59:42.413284+0000","last_became_active":"2024-08-15T16:59:42.407049+0000","last_became_peered":"2024-08-15T16:59:42.407049+0000","last_unstale":"2024-08-15T16:59:42.413284+0000","last_undegraded":"2024-08-15T16:59:42.413284+0000","last_fullsized":"2024-08-15T16:59:42.413284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418920+0000","last_change":"2024-08-15T16:59:41.373342+0000","last_active":"2024-08-15T16:59:42.418920+0000","last_peered":"2024-08-15T16:59:42.418920+0000","last_clean":"2024-08-15T16:59:42.418920+0000","last_became_active":"2024-08-15T16:59:41.373010+0000","last_became_peered":"2024-08-15T16:59:41.373010+0000","last_unstale":"2024-08-15T16:59:42.418920+0000","last_undegraded":"2024-08-15T16:59:42.418920+0000","last_fullsized":"2024-08-15T16:59:42.418920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413438+0000","last_change":"2024-08-15T16:59:42.404605+0000","last_active":"2024-08-15T16:59:42.413438+0000","last_peered":"2024-08-15T16:59:42.413438+0000","last_clean":"2024-08-15T16:59:42.413438+0000","last_became_active":"2024-08-15T16:59:42.404054+0000","last_became_peered":"2024-08-15T16:59:42.404054+0000","last_unstale":"2024-08-15T16:59:42.413438+0000","last_undegraded":"2024-08-15T16:59:42.413438+0000","last_fullsized":"2024-08-15T16:59:42.413438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419684+0000","last_change":"2024-08-15T16:59:41.381959+0000","last_active":"2024-08-15T16:59:42.419684+0000","last_peered":"2024-08-15T16:59:42.419684+0000","last_clean":"2024-08-15T16:59:42.419684+0000","last_became_active":"2024-08-15T16:59:41.381634+0000","last_became_peered":"2024-08-15T16:59:41.381634+0000","last_unstale":"2024-08-15T16:59:42.419684+0000","last_undegraded":"2024-08-15T16:59:42.419684+0000","last_fullsized":"2024-08-15T16:59:42.419684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413316+0000","last_change":"2024-08-15T16:59:42.408082+0000","last_active":"2024-08-15T16:59:42.413316+0000","last_peered":"2024-08-15T16:59:42.413316+0000","last_clean":"2024-08-15T16:59:42.413316+0000","last_became_active":"2024-08-15T16:59:42.407840+0000","last_became_peered":"2024-08-15T16:59:42.407840+0000","last_unstale":"2024-08-15T16:59:42.413316+0000","last_undegraded":"2024-08-15T16:59:42.413316+0000","last_fullsized":"2024-08-15T16:59:42.413316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:32.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422292+0000","last_change":"2024-08-15T16:59:41.381232+0000","last_active":"2024-08-15T16:59:42.422292+0000","last_peered":"2024-08-15T16:59:42.422292+0000","last_clean":"2024-08-15T16:59:42.422292+0000","last_became_active":"2024-08-15T16:59:41.380497+0000","last_became_peered":"2024-08-15T16:59:41.380497+0000","last_unstale":"2024-08-15T16:59:42.422292+0000","last_undegraded":"2024-08-15T16:59:42.422292+0000","last_fullsized":"2024-08-15T16:59:42.422292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414529+0000","last_change":"2024-08-15T16:59:42.410719+0000","last_active":"2024-08-15T16:59:42.414529+0000","last_peered":"2024-08-15T16:59:42.414529+0000","last_clean":"2024-08-15T16:59:42.414529+0000","last_became_active":"2024-08-15T16:59:42.410323+0000","last_became_peered":"2024-08-15T16:59:42.410323+0000","last_unstale":"2024-08-15T16:59:42.414529+0000","last_undegraded":"2024-08-15T16:59:42.414529+0000","last_fullsized":"2024-08-15T16:59:42.414529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:22:43.534462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387934+0000","last_change":"2024-08-15T16:59:41.375212+0000","last_active":"2024-08-15T16:59:42.387934+0000","last_peered":"2024-08-15T16:59:42.387934+0000","last_clean":"2024-08-15T16:59:42.387934+0000","last_became_active":"2024-08-15T16:59:41.374811+0000","last_became_peered":"2024-08-15T16:59:41.374811+0000","last_unstale":"2024-08-15T16:59:42.387934+0000","last_undegraded":"2024-08-15T16:59:42.387934+0000","last_fullsized":"2024-08-15T16:59:42.387934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419483+0000","last_change":"2024-08-15T16:59:42.416559+0000","last_active":"2024-08-15T16:59:42.419483+0000","last_peered":"2024-08-15T16:59:42.419483+0000","last_clean":"2024-08-15T16:59:42.419483+0000","last_became_active":"2024-08-15T16:59:42.416241+0000","last_became_peered":"2024-08-15T16:59:42.416241+0000","last_unstale":"2024-08-15T16:59:42.419483+0000","last_undegraded":"2024-08-15T16:59:42.419483+0000","last_fullsized":"2024-08-15T16:59:42.419483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:30:00.824354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395661+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:42.395661+0000","last_peered":"2024-08-15T16:59:42.395661+0000","last_clean":"2024-08-15T16:59:42.395661+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:42.395661+0000","last_undegraded":"2024-08-15T16:59:42.395661+0000","last_fullsized":"2024-08-15T16:59:42.395661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.386944+0000","last_change":"2024-08-15T16:59:41.374273+0000","last_active":"2024-08-15T16:59:42.386944+0000","last_peered":"2024-08-15T16:59:42.386944+0000","last_clean":"2024-08-15T16:59:42.386944+0000","last_became_active":"2024-08-15T16:59:41.373947+0000","last_became_peered":"2024-08-15T16:59:41.373947+0000","last_unstale":"2024-08-15T16:59:42.386944+0000","last_undegraded":"2024-08-15T16:59:42.386944+0000","last_fullsized":"2024-08-15T16:59:42.386944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:59:06.484884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417638+0000","last_change":"2024-08-15T16:59:42.412036+0000","last_active":"2024-08-15T16:59:42.417638+0000","last_peered":"2024-08-15T16:59:42.417638+0000","last_clean":"2024-08-15T16:59:42.417638+0000","last_became_active":"2024-08-15T16:59:42.411769+0000","last_became_peered":"2024-08-15T16:59:42.411769+0000","last_unstale":"2024-08-15T16:59:42.417638+0000","last_undegraded":"2024-08-15T16:59:42.417638+0000","last_fullsized":"2024-08-15T16:59:42.417638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395564+0000","last_change":"2024-08-15T16:59:41.384227+0000","last_active":"2024-08-15T16:59:42.395564+0000","last_peered":"2024-08-15T16:59:42.395564+0000","last_clean":"2024-08-15T16:59:42.395564+0000","last_became_active":"2024-08-15T16:59:41.384013+0000","last_became_peered":"2024-08-15T16:59:41.384013+0000","last_unstale":"2024-08-15T16:59:42.395564+0000","last_undegraded":"2024-08-15T16:59:42.395564+0000","last_fullsized":"2024-08-15T16:59:42.395564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417479+0000","last_change":"2024-08-15T16:59:42.406032+0000","last_active":"2024-08-15T16:59:42.417479+0000","last_peered":"2024-08-15T16:59:42.417479+0000","last_clean":"2024-08-15T16:59:42.417479+0000","last_became_active":"2024-08-15T16:59:42.405742+0000","last_became_peered":"2024-08-15T16:59:42.405742+0000","last_unstale":"2024-08-15T16:59:42.417479+0000","last_undegraded":"2024-08-15T16:59:42.417479+0000","last_fullsized":"2024-08-15T16:59:42.417479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387905+0000","last_change":"2024-08-15T16:59:41.382128+0000","last_active":"2024-08-15T16:59:42.387905+0000","last_peered":"2024-08-15T16:59:42.387905+0000","last_clean":"2024-08-15T16:59:42.387905+0000","last_became_active":"2024-08-15T16:59:41.381700+0000","last_became_peered":"2024-08-15T16:59:41.381700+0000","last_unstale":"2024-08-15T16:59:42.387905+0000","last_undegraded":"2024-08-15T16:59:42.387905+0000","last_fullsized":"2024-08-15T16:59:42.387905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414769+0000","last_change":"2024-08-15T16:59:42.407654+0000","last_active":"2024-08-15T16:59:42.414769+0000","last_peered":"2024-08-15T16:59:42.414769+0000","last_clean":"2024-08-15T16:59:42.414769+0000","last_became_active":"2024-08-15T16:59:42.405202+0000","last_became_peered":"2024-08-15T16:59:42.405202+0000","last_unstale":"2024-08-15T16:59:42.414769+0000","last_undegraded":"2024-08-15T16:59:42.414769+0000","last_fullsized":"2024-08-15T16:59:42.414769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391657+0000","last_change":"2024-08-15T16:59:41.374124+0000","last_active":"2024-08-15T16:59:42.391657+0000","last_peered":"2024-08-15T16:59:42.391657+0000","last_clean":"2024-08-15T16:59:42.391657+0000","last_became_active":"2024-08-15T16:59:41.373620+0000","last_became_peered":"2024-08-15T16:59:41.373620+0000","last_unstale":"2024-08-15T16:59:42.391657+0000","last_undegraded":"2024-08-15T16:59:42.391657+0000","last_fullsized":"2024-08-15T16:59:42.391657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:55:35.560885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.410018+0000","last_change":"2024-08-15T16:59:42.408577+0000","last_active":"2024-08-15T16:59:42.410018+0000","last_peered":"2024-08-15T16:59:42.410018+0000","last_clean":"2024-08-15T16:59:42.410018+0000","last_became_active":"2024-08-15T16:59:42.408272+0000","last_became_peered":"2024-08-15T16:59:42.408272+0000","last_unstale":"2024-08-15T16:59:42.410018+0000","last_undegraded":"2024-08-15T16:59:42.410018+0000","last_fullsized":"2024-08-15T16:59:42.410018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:55.794089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.396515+0000","last_change":"2024-08-15T16:59:41.383124+0000","last_active":"2024-08-15T16:59:42.396515+0000","last_peered":"2024-08-15T16:59:42.396515+0000","last_clean":"2024-08-15T16:59:42.396515+0000","last_became_active":"2024-08-15T16:59:41.382696+0000","last_became_peered":"2024-08-15T16:59:41.382696+0000","last_unstale":"2024-08-15T16:59:42.396515+0000","last_undegraded":"2024-08-15T16:59:42.396515+0000","last_fullsized":"2024-08-15T16:59:42.396515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414332+0000","last_change":"2024-08-15T16:59:42.406595+0000","last_active":"2024-08-15T16:59:42.414332+0000","last_peered":"2024-08-15T16:59:42.414332+0000","last_clean":"2024-08-15T16:59:42.414332+0000","last_became_active":"2024-08-15T16:59:42.406353+0000","last_became_peered":"2024-08-15T16:59:42.406353+0000","last_unstale":"2024-08-15T16:59:42.414332+0000","last_undegraded":"2024-08-15T16:59:42.414332+0000","last_fullsized":"2024-08-15T16:59:42.414332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:25:43.883310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398385+0000","last_change":"2024-08-15T16:59:41.381997+0000","last_active":"2024-08-15T16:59:42.398385+0000","last_peered":"2024-08-15T16:59:42.398385+0000","last_clean":"2024-08-15T16:59:42.398385+0000","last_became_active":"2024-08-15T16:59:41.381679+0000","last_became_peered":"2024-08-15T16:59:41.381679+0000","last_unstale":"2024-08-15T16:59:42.398385+0000","last_undegraded":"2024-08-15T16:59:42.398385+0000","last_fullsized":"2024-08-15T16:59:42.398385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:39.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411157+0000","last_change":"2024-08-15T16:59:42.404468+0000","last_active":"2024-08-15T16:59:42.411157+0000","last_peered":"2024-08-15T16:59:42.411157+0000","last_clean":"2024-08-15T16:59:42.411157+0000","last_became_active":"2024-08-15T16:59:42.403172+0000","last_became_peered":"2024-08-15T16:59:42.403172+0000","last_unstale":"2024-08-15T16:59:42.411157+0000","last_undegraded":"2024-08-15T16:59:42.411157+0000","last_fullsized":"2024-08-15T16:59:42.411157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.397905+0000","last_change":"2024-08-15T16:59:41.379177+0000","last_active":"2024-08-15T16:59:42.397905+0000","last_peered":"2024-08-15T16:59:42.397905+0000","last_clean":"2024-08-15T16:59:42.397905+0000","last_became_active":"2024-08-15T16:59:41.378909+0000","last_became_peered":"2024-08-15T16:59:41.378909+0000","last_unstale":"2024-08-15T16:59:42.397905+0000","last_undegraded":"2024-08-15T16:59:42.397905+0000","last_fullsized":"2024-08-15T16:59:42.397905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416728+0000","last_change":"2024-08-15T16:59:42.407782+0000","last_active":"2024-08-15T16:59:42.416728+0000","last_peered":"2024-08-15T16:59:42.416728+0000","last_clean":"2024-08-15T16:59:42.416728+0000","last_became_active":"2024-08-15T16:59:42.407483+0000","last_became_peered":"2024-08-15T16:59:42.407483+0000","last_unstale":"2024-08-15T16:59:42.416728+0000","last_undegraded":"2024-08-15T16:59:42.416728+0000","last_fullsized":"2024-08-15T16:59:42.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420095+0000","last_change":"2024-08-15T16:59:41.379440+0000","last_active":"2024-08-15T16:59:42.420095+0000","last_peered":"2024-08-15T16:59:42.420095+0000","last_clean":"2024-08-15T16:59:42.420095+0000","last_became_active":"2024-08-15T16:59:41.379116+0000","last_became_peered":"2024-08-15T16:59:41.379116+0000","last_unstale":"2024-08-15T16:59:42.420095+0000","last_undegraded":"2024-08-15T16:59:42.420095+0000","last_fullsized":"2024-08-15T16:59:42.420095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420125+0000","last_change":"2024-08-15T16:59:42.414998+0000","last_active":"2024-08-15T16:59:42.420125+0000","last_peered":"2024-08-15T16:59:42.420125+0000","last_clean":"2024-08-15T16:59:42.420125+0000","last_became_active":"2024-08-15T16:59:42.414696+0000","last_became_peered":"2024-08-15T16:59:42.414696+0000","last_unstale":"2024-08-15T16:59:42.420125+0000","last_undegraded":"2024-08-15T16:59:42.420125+0000","last_fullsized":"2024-08-15T16:59:42.420125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:42:39.871776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395563+0000","last_change":"2024-08-15T16:59:41.381044+0000","last_active":"2024-08-15T16:59:42.395563+0000","last_peered":"2024-08-15T16:59:42.395563+0000","last_clean":"2024-08-15T16:59:42.395563+0000","last_became_active":"2024-08-15T16:59:41.380776+0000","last_became_peered":"2024-08-15T16:59:41.380776+0000","last_unstale":"2024-08-15T16:59:42.395563+0000","last_undegraded":"2024-08-15T16:59:42.395563+0000","last_fullsized":"2024-08-15T16:59:42.395563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:21:56.894684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.409831+0000","last_change":"2024-08-15T16:59:42.408779+0000","last_active":"2024-08-15T16:59:42.409831+0000","last_peered":"2024-08-15T16:59:42.409831+0000","last_clean":"2024-08-15T16:59:42.409831+0000","last_became_active":"2024-08-15T16:59:42.408345+0000","last_became_peered":"2024-08-15T16:59:42.408345+0000","last_unstale":"2024-08-15T16:59:42.409831+0000","last_undegraded":"2024-08-15T16:59:42.409831+0000","last_fullsized":"2024-08-15T16:59:42.409831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:55:03.519854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395271+0000","last_change":"2024-08-15T16:59:41.382297+0000","last_active":"2024-08-15T16:59:42.395271+0000","last_peered":"2024-08-15T16:59:42.395271+0000","last_clean":"2024-08-15T16:59:42.395271+0000","last_became_active":"2024-08-15T16:59:41.381812+0000","last_became_peered":"2024-08-15T16:59:41.381812+0000","last_unstale":"2024-08-15T16:59:42.395271+0000","last_undegraded":"2024-08-15T16:59:42.395271+0000","last_fullsized":"2024-08-15T16:59:42.395271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418823+0000","last_change":"2024-08-15T16:59:42.402821+0000","last_active":"2024-08-15T16:59:42.418823+0000","last_peered":"2024-08-15T16:59:42.418823+0000","last_clean":"2024-08-15T16:59:42.418823+0000","last_became_active":"2024-08-15T16:59:42.402044+0000","last_became_peered":"2024-08-15T16:59:42.402044+0000","last_unstale":"2024-08-15T16:59:42.418823+0000","last_undegraded":"2024-08-15T16:59:42.418823+0000","last_fullsized":"2024-08-15T16:59:42.418823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395855+0000","last_change":"2024-08-15T16:59:41.383793+0000","last_active":"2024-08-15T16:59:42.395855+0000","last_peered":"2024-08-15T16:59:42.395855+0000","last_clean":"2024-08-15T16:59:42.395855+0000","last_became_active":"2024-08-15T16:59:41.383431+0000","last_became_peered":"2024-08-15T16:59:41.383431+0000","last_unstale":"2024-08-15T16:59:42.395855+0000","last_undegraded":"2024-08-15T16:59:42.395855+0000","last_fullsized":"2024-08-15T16:59:42.395855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422596+0000","last_change":"2024-08-15T16:59:42.419454+0000","last_active":"2024-08-15T16:59:42.422596+0000","last_peered":"2024-08-15T16:59:42.422596+0000","last_clean":"2024-08-15T16:59:42.422596+0000","last_became_active":"2024-08-15T16:59:42.419157+0000","last_became_peered":"2024-08-15T16:59:42.419157+0000","last_unstale":"2024-08-15T16:59:42.422596+0000","last_undegraded":"2024-08-15T16:59:42.422596+0000","last_fullsized":"2024-08-15T16:59:42.422596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421197+0000","last_change":"2024-08-15T16:59:41.382836+0000","last_active":"2024-08-15T16:59:42.421197+0000","last_peered":"2024-08-15T16:59:42.421197+0000","last_clean":"2024-08-15T16:59:42.421197+0000","last_became_active":"2024-08-15T16:59:41.382595+0000","last_became_peered":"2024-08-15T16:59:41.382595+0000","last_unstale":"2024-08-15T16:59:42.421197+0000","last_undegraded":"2024-08-15T16:59:42.421197+0000","last_fullsized":"2024-08-15T16:59:42.421197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401292+0000","last_change":"2024-08-15T16:59:42.399359+0000","last_active":"2024-08-15T16:59:42.401292+0000","last_peered":"2024-08-15T16:59:42.401292+0000","last_clean":"2024-08-15T16:59:42.401292+0000","last_became_active":"2024-08-15T16:59:42.398800+0000","last_became_peered":"2024-08-15T16:59:42.398800+0000","last_unstale":"2024-08-15T16:59:42.401292+0000","last_undegraded":"2024-08-15T16:59:42.401292+0000","last_fullsized":"2024-08-15T16:59:42.401292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.397841+0000","last_change":"2024-08-15T16:59:42.396158+0000","last_active":"2024-08-15T16:59:42.397841+0000","last_peered":"2024-08-15T16:59:42.397841+0000","last_clean":"2024-08-15T16:59:42.397841+0000","last_became_active":"2024-08-15T16:59:42.395850+0000","last_became_peered":"2024-08-15T16:59:42.395850+0000","last_unstale":"2024-08-15T16:59:42.397841+0000","last_undegraded":"2024-08-15T16:59:42.397841+0000","last_fullsized":"2024-08-15T16:59:42.397841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:39.678214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422026+0000","last_change":"2024-08-15T16:59:41.385558+0000","last_active":"2024-08-15T16:59:42.422026+0000","last_peered":"2024-08-15T16:59:42.422026+0000","last_clean":"2024-08-15T16:59:42.422026+0000","last_became_active":"2024-08-15T16:59:41.385303+0000","last_became_peered":"2024-08-15T16:59:41.385303+0000","last_unstale":"2024-08-15T16:59:42.422026+0000","last_undegraded":"2024-08-15T16:59:42.422026+0000","last_fullsized":"2024-08-15T16:59:42.422026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400920+0000","last_change":"2024-08-15T16:59:42.396765+0000","last_active":"2024-08-15T16:59:42.400920+0000","last_peered":"2024-08-15T16:59:42.400920+0000","last_clean":"2024-08-15T16:59:42.400920+0000","last_became_active":"2024-08-15T16:59:42.396133+0000","last_became_peered":"2024-08-15T16:59:42.396133+0000","last_unstale":"2024-08-15T16:59:42.400920+0000","last_undegraded":"2024-08-15T16:59:42.400920+0000","last_fullsized":"2024-08-15T16:59:42.400920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:38:03.347839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420319+0000","last_change":"2024-08-15T16:59:41.381180+0000","last_active":"2024-08-15T16:59:42.420319+0000","last_peered":"2024-08-15T16:59:42.420319+0000","last_clean":"2024-08-15T16:59:42.420319+0000","last_became_active":"2024-08-15T16:59:41.381011+0000","last_became_peered":"2024-08-15T16:59:41.381011+0000","last_unstale":"2024-08-15T16:59:42.420319+0000","last_undegraded":"2024-08-15T16:59:42.420319+0000","last_fullsized":"2024-08-15T16:59:42.420319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420390+0000","last_change":"2024-08-15T16:59:42.416366+0000","last_active":"2024-08-15T16:59:42.420390+0000","last_peered":"2024-08-15T16:59:42.420390+0000","last_clean":"2024-08-15T16:59:42.420390+0000","last_became_active":"2024-08-15T16:59:42.415630+0000","last_became_peered":"2024-08-15T16:59:42.415630+0000","last_unstale":"2024-08-15T16:59:42.420390+0000","last_undegraded":"2024-08-15T16:59:42.420390+0000","last_fullsized":"2024-08-15T16:59:42.420390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:23.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420472+0000","last_change":"2024-08-15T16:59:41.379215+0000","last_active":"2024-08-15T16:59:42.420472+0000","last_peered":"2024-08-15T16:59:42.420472+0000","last_clean":"2024-08-15T16:59:42.420472+0000","last_became_active":"2024-08-15T16:59:41.378905+0000","last_became_peered":"2024-08-15T16:59:41.378905+0000","last_unstale":"2024-08-15T16:59:42.420472+0000","last_undegraded":"2024-08-15T16:59:42.420472+0000","last_fullsized":"2024-08-15T16:59:42.420472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416924+0000","last_change":"2024-08-15T16:59:42.408126+0000","last_active":"2024-08-15T16:59:42.416924+0000","last_peered":"2024-08-15T16:59:42.416924+0000","last_clean":"2024-08-15T16:59:42.416924+0000","last_became_active":"2024-08-15T16:59:42.407836+0000","last_became_peered":"2024-08-15T16:59:42.407836+0000","last_unstale":"2024-08-15T16:59:42.416924+0000","last_undegraded":"2024-08-15T16:59:42.416924+0000","last_fullsized":"2024-08-15T16:59:42.416924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421366+0000","last_change":"2024-08-15T16:59:41.382335+0000","last_active":"2024-08-15T16:59:42.421366+0000","last_peered":"2024-08-15T16:59:42.421366+0000","last_clean":"2024-08-15T16:59:42.421366+0000","last_became_active":"2024-08-15T16:59:41.382004+0000","last_became_peered":"2024-08-15T16:59:41.382004+0000","last_unstale":"2024-08-15T16:59:42.421366+0000","last_undegraded":"2024-08-15T16:59:42.421366+0000","last_fullsized":"2024-08-15T16:59:42.421366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":811008,"data_stored":664727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":217088,"data_stored":73879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":73915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.794}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":212992,"data_stored":73789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.484}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":663822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:50.114 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T16:59:50.114 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-15T16:59:50.114 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-15T16:59:50.114 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-15T16:59:50.115 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-15T16:59:50.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.209+0000 7f02e889b700 1 Processor -- start 2024-08-15T16:59:50.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.209+0000 7f02e889b700 1 -- start start 2024-08-15T16:59:50.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f02e40cb390 0x7f02e4160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40cbaa0 con 0x7f02e4154900 2024-08-15T16:59:50.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e4068790 con 0x7f02e40cb390 2024-08-15T16:59:50.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e41583f0 con 0x7f02e40b14a0 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e359e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f02e40cb390 0x7f02e4160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59464/0 (socket says 172.21.15.146:59464) 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 -- 172.21.15.146:0/3783142181 learned_addr learned my addr 172.21.15.146:0/3783142181 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 -- 172.21.15.146:0/3783142181 >> v2:172.21.15.161:3300/0 conn(0x7f02e40cb390 msgr2=0x7f02e4160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 --2- 172.21.15.146:0/3783142181 >> v2:172.21.15.161:3300/0 conn(0x7f02e40cb390 0x7f02e4160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 -- 172.21.15.146:0/3783142181 >> v1:172.21.15.161:6789/0 conn(0x7f02e4154900 legacy=0x7f02e40b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 -- 172.21.15.146:0/3783142181 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4157630 con 0x7f02e40b14a0 2024-08-15T16:59:50.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02da59c700 1 --2- 172.21.15.146:0/3783142181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0099b0 tx=0x7f02cc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2b716dcb9a22555 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e259c700 1 -- 172.21.15.146:0/3783142181 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc009ea0 con 0x7f02e40b14a0 2024-08-15T16:59:50.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e259c700 1 -- 172.21.15.146:0/3783142181 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc00b860 con 0x7f02e40b14a0 2024-08-15T16:59:50.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e259c700 1 -- 172.21.15.146:0/3783142181 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00bc00 con 0x7f02e40b14a0 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- 172.21.15.146:0/3783142181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 msgr2=0x7f02e40cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 --2- 172.21.15.146:0/3783142181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0099b0 tx=0x7f02cc009250 comp rx=0 tx=0).stop 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- 172.21.15.146:0/3783142181 shutdown_connections 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 --2- 172.21.15.146:0/3783142181 >> v2:172.21.15.161:3300/0 conn(0x7f02e40cb390 0x7f02e4160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 --2- 172.21.15.146:0/3783142181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40b14a0 0x7f02e40cae50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- 172.21.15.146:0/3783142181 >> 172.21.15.146:0/3783142181 conn(0x7f02e40678d0 msgr2=0x7f02e4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- 172.21.15.146:0/3783142181 shutdown_connections 2024-08-15T16:59:50.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.213+0000 7f02e889b700 1 -- 172.21.15.146:0/3783142181 wait complete. 2024-08-15T16:59:50.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 Processor -- start 2024-08-15T16:59:50.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- start start 2024-08-15T16:59:50.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02e40b14a0 0x7f02e4141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40cb390 0x7f02e40a9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40a9aa0 con 0x7f02e4154900 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4168520 con 0x7f02e40b14a0 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4168770 con 0x7f02e40cb390 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49602/0 (socket says 172.21.15.146:49602) 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 -- 172.21.15.146:0/3209286666 learned_addr learned my addr 172.21.15.146:0/3209286666 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02da59c700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40cb390 0x7f02e40a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e2d9d700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02e40b14a0 0x7f02e4141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 -- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40cb390 msgr2=0x7f02e40a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40cb390 0x7f02e40a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 -- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02e40b14a0 msgr2=0x7f02e4141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02e40b14a0 0x7f02e4141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 -- 172.21.15.146:0/3209286666 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc0094f0 con 0x7f02e4154900 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e359e700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009df0 tx=0x7f02dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00caa0 con 0x7f02e4154900 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc00f040 con 0x7f02e4154900 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e4168a50 con 0x7f02e4154900 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00ce70 con 0x7f02e4154900 2024-08-15T16:59:50.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.217+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4169010 con 0x7f02e4154900 2024-08-15T16:59:50.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e40643a0 con 0x7f02e4154900 2024-08-15T16:59:50.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f02dc014690 con 0x7f02e4154900 2024-08-15T16:59:50.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02db7fe700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 0x7f02c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02dc07d3c0 con 0x7f02e4154900 2024-08-15T16:59:50.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02e2d9d700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 0x7f02c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.221+0000 7f02e2d9d700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 0x7f02c4062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02d4003aa0 tx=0x7f02d4000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.225+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f02dc0512a0 con 0x7f02e4154900 2024-08-15T16:59:50.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.429+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f02e406f5f0 con 0x7f02e4154900 2024-08-15T16:59:50.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.433+0000 7f02db7fe700 1 -- 172.21.15.146:0/3209286666 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+154541 (secure 0 0 0) 0x7f02dc050e30 con 0x7f02e4154900 2024-08-15T16:59:50.449 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:50.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 msgr2=0x7f02c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 0x7f02c4062d10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f02d4003aa0 tx=0x7f02d4000bc0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 msgr2=0x7f02e40a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f02dc009df0 tx=0x7f02dc00b040 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 shutdown_connections 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f02c4060860 0x7f02c4062d10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f02e4154900 0x7f02e40a9560 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f02e40cb390 0x7f02e40a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 --2- 172.21.15.146:0/3209286666 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02e40b14a0 0x7f02e4141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 >> 172.21.15.146:0/3209286666 conn(0x7f02e40678d0 msgr2=0x7f02e40ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 shutdown_connections 2024-08-15T16:59:50.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.437+0000 7f02e889b700 1 -- 172.21.15.146:0/3209286666 wait complete. 2024-08-15T16:59:50.463 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":24128},{"type":"v1","addr":"172.21.15.161:6833","nonce":24128}]},"active_addr":"172.21.15.161:6833/24128","active_change":"2024-08-15T16:59:29.630371+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4138,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2403879965}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2083146000}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2356192191}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":585897228}]}]} 2024-08-15T16:59:50.466 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-15T16:59:50.467 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-15T16:59:50.467 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:50.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.557+0000 7f39df160700 1 Processor -- start 2024-08-15T16:59:50.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.557+0000 7f39df160700 1 -- start start 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f39d80b0350 0x7f39d80ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d806df10 0x7f39d806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d806e8b0 con 0x7f39d8155d20 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80b07a0 con 0x7f39d80b0350 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80af2b0 con 0x7f39d806df10 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39de15e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f39d8155d20 0x7f39d80aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60050/0 (socket says 172.21.15.146:60050) 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39de15e700 1 -- 172.21.15.146:0/4234671594 learned_addr learned my addr 172.21.15.146:0/4234671594 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:50.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 --2- 172.21.15.146:0/4234671594 >> v2:172.21.15.161:3300/0 conn(0x7f39d80b0350 0x7f39d80ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39de95f700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d806df10 0x7f39d806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365858528 0 0) 0x7f39d4001110 con 0x7f39d8155d20 2024-08-15T16:59:50.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c4003570 con 0x7f39d8155d20 2024-08-15T16:59:50.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 -- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d806df10 msgr2=0x7f39d806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:50.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d806df10 0x7f39d806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 -- 172.21.15.146:0/4234671594 >> v1:172.21.15.146:6789/0 conn(0x7f39d8155d20 legacy=0x7f39d80aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2292407176 0 0) 0x7f39c4003570 con 0x7f39d8155d20 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 -- 172.21.15.146:0/4234671594 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80af590 con 0x7f39d80b0350 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd95d700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d80b0350 0x7f39d80ad7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098e0 tx=0x7f39c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95816a33f0c3b3ee server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4001600 con 0x7f39d80b0350 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c8011420 con 0x7f39d80b0350 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39dd15c700 1 -- 172.21.15.146:0/4234671594 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80117c0 con 0x7f39d80b0350 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d80b0350 msgr2=0x7f39d80ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d80b0350 0x7f39d80ad7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39c80098e0 tx=0x7f39c80091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- 172.21.15.146:0/4234671594 shutdown_connections 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d806df10 0x7f39d806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 --2- 172.21.15.146:0/4234671594 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d80b0350 0x7f39d80ad7c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- 172.21.15.146:0/4234671594 >> 172.21.15.146:0/4234671594 conn(0x7f39d80678d0 msgr2=0x7f39d8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- 172.21.15.146:0/4234671594 shutdown_connections 2024-08-15T16:59:50.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.561+0000 7f39df160700 1 -- 172.21.15.146:0/4234671594 wait complete. 2024-08-15T16:59:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 Processor -- start 2024-08-15T16:59:50.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- start start 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d80b0350 0x7f39d8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d8155d20 0x7f39d8169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8158b60 con 0x7f39d806df10 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8156630 con 0x7f39d8155d20 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8157f50 con 0x7f39d80b0350 2024-08-15T16:59:50.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49610/0 (socket says 172.21.15.146:49610) 2024-08-15T16:59:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 -- 172.21.15.146:0/326640095 learned_addr learned my addr 172.21.15.146:0/326640095 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39dd95d700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d80b0350 0x7f39d8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de95f700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d8155d20 0x7f39d8169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 -- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d80b0350 msgr2=0x7f39d8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d80b0350 0x7f39d8158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 -- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d8155d20 msgr2=0x7f39d8169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d8155d20 0x7f39d8169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 -- 172.21.15.146:0/326640095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8009420 con 0x7f39d806df10 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39de15e700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39d400cb90 tx=0x7f39d4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4007150 con 0x7f39d806df10 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d40072d0 con 0x7f39d806df10 2024-08-15T16:59:50.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d8158230 con 0x7f39d806df10 2024-08-15T16:59:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4007690 con 0x7f39d806df10 2024-08-15T16:59:50.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.565+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d81587f0 con 0x7f39d806df10 2024-08-15T16:59:50.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d80643a0 con 0x7f39d806df10 2024-08-15T16:59:50.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f39d4004a40 con 0x7f39d806df10 2024-08-15T16:59:50.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39ce7fc700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 0x7f39bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f39d407c120 con 0x7f39d806df10 2024-08-15T16:59:50.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39dd95d700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 0x7f39bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.569+0000 7f39dd95d700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 0x7f39bc062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39c80030c0 tx=0x7f39c8016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.577+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f39d404c5f0 con 0x7f39d806df10 2024-08-15T16:59:50.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.741+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39d806e8b0 con 0x7f39d806df10 2024-08-15T16:59:50.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.741+0000 7f39ce7fc700 1 -- 172.21.15.146:0/326640095 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f39d806e8b0 con 0x7f39d806df10 2024-08-15T16:59:50.755 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:50.755 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 msgr2=0x7f39bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 0x7f39bc062d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39c80030c0 tx=0x7f39c8016000 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 msgr2=0x7f39d81560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39d400cb90 tx=0x7f39d4001e90 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 shutdown_connections 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f39bc060860 0x7f39bc062d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39d8155d20 0x7f39d8169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39d80b0350 0x7f39d8158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 --2- 172.21.15.146:0/326640095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39d806df10 0x7f39d81560f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 >> 172.21.15.146:0/326640095 conn(0x7f39d80678d0 msgr2=0x7f39d8063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 shutdown_connections 2024-08-15T16:59:50.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.745+0000 7f39df160700 1 -- 172.21.15.146:0/326640095 wait complete. 2024-08-15T16:59:50.770 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-15T16:59:50.770 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:50.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 Processor -- start 2024-08-15T16:59:50.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 -- start start 2024-08-15T16:59:50.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 0x7efddc0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efddc0b12d0 0x7efddc0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0cbaa0 con 0x7efddc0cb390 2024-08-15T16:59:50.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc068790 con 0x7efddc0b12d0 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddb59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc1583f0 con 0x7efddc154900 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efddc0b12d0 0x7efddc0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdda59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 0x7efddc0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efddc0b12d0 0x7efddc0cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:33956/0 (socket says 172.21.15.146:33956) 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 -- 172.21.15.146:0/1892337659 learned_addr learned my addr 172.21.15.146:0/1892337659 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:50.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efddad9d700 1 --1- 172.21.15.146:0/1892337659 >> v1:172.21.15.161:6789/0 conn(0x7efddc0cb390 0x7efddc06df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42042/0 (socket says 172.21.15.146:42042) 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 -- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 msgr2=0x7efddc0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 0x7efddc0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 -- 172.21.15.146:0/1892337659 >> v1:172.21.15.161:6789/0 conn(0x7efddc0cb390 legacy=0x7efddc06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 -- 172.21.15.146:0/1892337659 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0b1700 con 0x7efddc0b12d0 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdda59c700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 0x7efddc0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:50.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd9d9b700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0b12d0 0x7efddc0cae50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdc40099b0 tx=0x7efdc4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210839aa41e34e52 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd959a700 1 -- 172.21.15.146:0/1892337659 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4009ea0 con 0x7efddc0b12d0 2024-08-15T16:59:50.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd959a700 1 -- 172.21.15.146:0/1892337659 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc400b860 con 0x7efddc0b12d0 2024-08-15T16:59:50.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.865+0000 7efdd959a700 1 -- 172.21.15.146:0/1892337659 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc400bc00 con 0x7efddc0b12d0 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0b12d0 msgr2=0x7efddc0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0b12d0 0x7efddc0cae50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efdc40099b0 tx=0x7efdc4009250 comp rx=0 tx=0).stop 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- 172.21.15.146:0/1892337659 shutdown_connections 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0b12d0 0x7efddc0cae50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- 172.21.15.146:0/1892337659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc154900 0x7efddc0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- 172.21.15.146:0/1892337659 >> 172.21.15.146:0/1892337659 conn(0x7efddc0678d0 msgr2=0x7efddc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:50.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- 172.21.15.146:0/1892337659 shutdown_connections 2024-08-15T16:59:50.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- 172.21.15.146:0/1892337659 wait complete. 2024-08-15T16:59:50.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 Processor -- start 2024-08-15T16:59:50.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- start start 2024-08-15T16:59:50.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 0x7efddc0c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0cb390 0x7efddc0b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0a9860 con 0x7efddc154900 2024-08-15T16:59:50.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0aa3e0 con 0x7efddc0cb390 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddb59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0c7c90 con 0x7efddc0b12d0 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efdda59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 0x7efddc0c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49622/0 (socket says 172.21.15.146:49622) 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efddad9d700 1 -- 172.21.15.146:0/2569271106 learned_addr learned my addr 172.21.15.146:0/2569271106 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:50.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.869+0000 7efdd9d9b700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0cb390 0x7efddc0b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 -- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 msgr2=0x7efddc0c7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 0x7efddc0c7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 -- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0cb390 msgr2=0x7efddc0b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:50.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0cb390 0x7efddc0b03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:50.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 -- 172.21.15.146:0/2569271106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc009420 con 0x7efddc154900 2024-08-15T16:59:50.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdda59c700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 0x7efddc0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:50.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddad9d700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdd400cd30 tx=0x7efdd4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400ce30 con 0x7efddc154900 2024-08-15T16:59:50.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd4005df0 con 0x7efddc154900 2024-08-15T16:59:50.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc40094f0 con 0x7efddc154900 2024-08-15T16:59:50.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd400ab00 con 0x7efddc154900 2024-08-15T16:59:50.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc0a9d90 con 0x7efddc154900 2024-08-15T16:59:50.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0aa550 con 0x7efddc154900 2024-08-15T16:59:50.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7efdd40070f0 con 0x7efddc154900 2024-08-15T16:59:50.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 0x7efdc0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:50.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.873+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efdd407cb10 con 0x7efddc154900 2024-08-15T16:59:50.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.877+0000 7efdda59c700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 0x7efdc0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:50.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.877+0000 7efdda59c700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 0x7efdc0062d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011600 tx=0x7efdcc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:50.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:50.881+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7efdd404cfe0 con 0x7efddc154900 2024-08-15T16:59:51.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.049+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efddc0643a0 con 0x7efddc154900 2024-08-15T16:59:51.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.049+0000 7efdd27fc700 1 -- 172.21.15.146:0/2569271106 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7efdd4050600 con 0x7efddc154900 2024-08-15T16:59:51.061 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:51.062 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:51.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 msgr2=0x7efdc0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 0x7efdc0062d40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011600 tx=0x7efdcc0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 msgr2=0x7efddc16a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efdd400cd30 tx=0x7efdd4001ad0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 shutdown_connections 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7efdc0060890 0x7efdc0062d40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efddc154900 0x7efddc16a3a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efddc0cb390 0x7efddc0b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 --2- 172.21.15.146:0/2569271106 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efddc0b12d0 0x7efddc0c7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 >> 172.21.15.146:0/2569271106 conn(0x7efddc0678d0 msgr2=0x7efddc06cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 shutdown_connections 2024-08-15T16:59:51.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.053+0000 7efddb59e700 1 -- 172.21.15.146:0/2569271106 wait complete. 2024-08-15T16:59:51.076 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-15T16:59:51.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-15T16:59:51.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-15T16:59:51.077 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-15T16:59:51.078 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-15T16:59:51.078 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-15T16:59:51.079 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-15T16:59:51.079 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-15T16:59:51.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 Processor -- start 2024-08-15T16:59:51.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 -- start start 2024-08-15T16:59:51.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9c0c13b630 0x7f9c0c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 0x7f9c0c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0c13cf90 con 0x7f9c0c0b1950 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0c13c380 con 0x7f9c0c13b630 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c1093f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c13c660 con 0x7f9c0c13bf10 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0ad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9c0c0b1950 0x7f9c0c158db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60060/0 (socket says 172.21.15.146:60060) 2024-08-15T16:59:51.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/1003722961 learned_addr learned my addr 172.21.15.146:0/1003722961 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 --2- 172.21.15.146:0/1003722961 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 0x7f9c0c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0a59c700 1 --2- 172.21.15.146:0/1003722961 >> v2:172.21.15.161:3300/0 conn(0x7f9c0c13b630 0x7f9c0c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0959a700 1 -- 172.21.15.146:0/1003722961 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534906660 0 0) 0x7f9c0c13cf90 con 0x7f9c0c0b1950 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0959a700 1 -- 172.21.15.146:0/1003722961 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf4003570 con 0x7f9c0c0b1950 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 -- 172.21.15.146:0/1003722961 >> v2:172.21.15.161:3300/0 conn(0x7f9c0c13b630 msgr2=0x7f9c0c0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.219 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 --2- 172.21.15.146:0/1003722961 >> v2:172.21.15.161:3300/0 conn(0x7f9c0c13b630 0x7f9c0c0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 -- 172.21.15.146:0/1003722961 >> v1:172.21.15.146:6789/0 conn(0x7f9c0c0b1950 legacy=0x7f9c0c158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 -- 172.21.15.146:0/1003722961 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c13c940 con 0x7f9c0c13bf10 2024-08-15T16:59:51.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f85990c7700 1 Processor -- start 2024-08-15T16:59:51.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f85990c7700 1 -- start start 2024-08-15T16:59:51.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f25e5d12700 1 Processor -- start 2024-08-15T16:59:51.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f3a4f59e700 1 Processor -- start 2024-08-15T16:59:51.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c09d9b700 1 --2- 172.21.15.146:0/1003722961 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 0x7f9c0c06df10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c000098e0 tx=0x7f9c000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a753174fe30bf8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f3a4f59e700 1 -- start start 2024-08-15T16:59:51.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f9c0959a700 1 -- 172.21.15.146:0/1003722961 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011630 con 0x7f9c0c13bf10 2024-08-15T16:59:51.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c0959a700 1 -- 172.21.15.146:0/1003722961 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00011790 con 0x7f9c0c13bf10 2024-08-15T16:59:51.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c0959a700 1 -- 172.21.15.146:0/1003722961 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00011b30 con 0x7f9c0c13bf10 2024-08-15T16:59:51.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 Processor -- start 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 -- start start 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85990c7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8594155d20 0x7f85940b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85990c7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85940aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85990c7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85940af2b0 con 0x7f859406df10 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85990c7700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85940adc10 con 0x7f8594155d20 2024-08-15T16:59:51.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85990c7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85940af590 con 0x7f85940ad7c0 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.205+0000 7f25e5d12700 1 -- start start 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e014f850 0x7f25e00691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25e014fc20 0x7f25e0156200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 0x7f25e00aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0069a40 con 0x7f25e00ae5d0 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0069d20 con 0x7f25e014fc20 2024-08-15T16:59:51.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e006a000 con 0x7f25e014f850 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 -- 172.21.15.146:0/1003722961 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 msgr2=0x7f9c0c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 --2- 172.21.15.146:0/1003722961 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 0x7f9c0c06df10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c000098e0 tx=0x7f9c000091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25e014fc20 0x7f25e0156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25e014fc20 0x7f25e0156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:33988/0 (socket says 172.21.15.146:33988) 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 -- 172.21.15.146:0/2586511456 learned_addr learned my addr 172.21.15.146:0/2586511456 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25df7fe700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e014f850 0x7f25e00691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25dffff700 1 --2- 172.21.15.146:0/2586511456 >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 0x7f25e00aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8594155d20 0x7f85940b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8594155d20 0x7f85940b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49630/0 (socket says 172.21.15.146:49630) 2024-08-15T16:59:51.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 -- 172.21.15.146:0/1635431464 learned_addr learned my addr 172.21.15.146:0/1635431464 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85988c6700 1 --1- 172.21.15.146:0/1635431464 >> v1:172.21.15.146:6789/0 conn(0x7f859406df10 0x7f85941599c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60064/0 (socket says 172.21.15.146:60064) 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f85937fe700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85940aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 -- 172.21.15.146:0/1635431464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 msgr2=0x7f85940aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85940aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 -- 172.21.15.146:0/1635431464 >> v1:172.21.15.146:6789/0 conn(0x7f859406df10 legacy=0x7f85941599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 Processor -- start 2024-08-15T16:59:51.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 -- 172.21.15.146:0/1003722961 shutdown_connections 2024-08-15T16:59:51.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 --2- 172.21.15.146:0/1003722961 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c13bf10 0x7f9c0c06df10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 --2- 172.21.15.146:0/1003722961 >> v2:172.21.15.161:3300/0 conn(0x7f9c0c13b630 0x7f9c0c0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 -- 172.21.15.146:0/1003722961 >> 172.21.15.146:0/1003722961 conn(0x7f9c0c0678d0 msgr2=0x7f9c0c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 -- 172.21.15.146:0/1635431464 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85940af870 con 0x7f8594155d20 2024-08-15T16:59:51.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 Processor -- start 2024-08-15T16:59:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8593fff700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8594155d20 0x7f85940b0350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f858c009f20 tx=0x7f858c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8572f2d5657f84ff server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 -- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e014f850 msgr2=0x7f25e00691f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e014f850 0x7f25e00691f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 -- 172.21.15.146:0/2586511456 >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 msgr2=0x7f25e00aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 --2- 172.21.15.146:0/2586511456 >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 0x7f25e00aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 -- 172.21.15.146:0/2586511456 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0155ce0 con 0x7f25e014fc20 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 -- start start 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f34e0155140 0x7f34e0155510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f34e006b390 0x7f34e00aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e00aa890 con 0x7f34e006e2b0 2024-08-15T16:59:51.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e00a9fd0 con 0x7f34e006b390 2024-08-15T16:59:51.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34e5736700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e0159cb0 con 0x7f34e0155140 2024-08-15T16:59:51.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 -- start start 2024-08-15T16:59:51.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25dffff700 1 --2- 172.21.15.146:0/2586511456 >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 0x7f25e00aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:51.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25deffd700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014fc20 0x7f25e0156200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25e0065660 tx=0x7f25d8007820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c445cc1bf3a2ade9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 -- 172.21.15.146:0/1003722961 shutdown_connections 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a500ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4f59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3a5006df10 0x7f3a5006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4f59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a500af2b0 con 0x7f3a50155d20 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4f59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a500b07a0 con 0x7f3a5006df10 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4f59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5006e8d0 con 0x7f3a500b0350 2024-08-15T16:59:51.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4e59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3a50155d20 0x7f3a500aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42046/0 (socket says 172.21.15.146:42046) 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4e59c700 1 -- 172.21.15.146:0/3838399142 learned_addr learned my addr 172.21.15.146:0/3838399142 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/3838399142 >> v2:172.21.15.161:3300/0 conn(0x7f3a5006df10 0x7f3a5006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8592ffd700 1 -- 172.21.15.146:0/1635431464 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c00e070 con 0x7f8594155d20 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8592ffd700 1 -- 172.21.15.146:0/1635431464 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c00b8d0 con 0x7f8594155d20 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f8592ffd700 1 -- 172.21.15.146:0/1635431464 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c0092e0 con 0x7f8594155d20 2024-08-15T16:59:51.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f34e0155140 0x7f34e0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4dd9b700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a500ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f34e0155140 0x7f34e0155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34020/0 (socket says 172.21.15.146:34020) 2024-08-15T16:59:51.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 -- 172.21.15.146:0/3943448813 learned_addr learned my addr 172.21.15.146:0/3943448813 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25de7fc700 1 -- 172.21.15.146:0/2586511456 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8007bd0 con 0x7f25e014fc20 2024-08-15T16:59:51.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25de7fc700 1 -- 172.21.15.146:0/2586511456 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d8007d30 con 0x7f25e014fc20 2024-08-15T16:59:51.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25de7fc700 1 -- 172.21.15.146:0/2586511456 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8007ea0 con 0x7f25e014fc20 2024-08-15T16:59:51.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 -- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 msgr2=0x7f3a500ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a500ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 -- 172.21.15.146:0/3838399142 >> v1:172.21.15.161:6789/0 conn(0x7f3a50155d20 legacy=0x7f3a500aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 -- 172.21.15.146:0/3838399142 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a500af590 con 0x7f3a5006df10 2024-08-15T16:59:51.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 -- 172.21.15.146:0/3943448813 >> v2:172.21.15.146:3300/0 conn(0x7f34e006b390 msgr2=0x7f34e00aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 --2- 172.21.15.146:0/3943448813 >> v2:172.21.15.146:3300/0 conn(0x7f34e006b390 0x7f34e00aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 -- 172.21.15.146:0/3943448813 >> v1:172.21.15.161:6789/0 conn(0x7f34e006e2b0 legacy=0x7f34e0158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 -- 172.21.15.146:0/3943448813 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00aa240 con 0x7f34e0155140 2024-08-15T16:59:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a5006df10 0x7f3a5006e390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009e90 tx=0x7f3a48009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e98d5258d427d36 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc73815b240 0x7fc73806a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73815a920 con 0x7fc738151370 2024-08-15T16:59:51.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73806c2f0 con 0x7fc738151740 2024-08-15T16:59:51.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7fc73d4e7700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73806c780 con 0x7fc73815b240 2024-08-15T16:59:51.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f34deffd700 1 --2- 172.21.15.146:0/3943448813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e0155140 0x7f34e0155510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34e0068ba0 tx=0x7f34d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec43bd44f5870bb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f34ddffb700 1 -- 172.21.15.146:0/3943448813 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000b860 con 0x7f34e0155140 2024-08-15T16:59:51.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f34ddffb700 1 -- 172.21.15.146:0/3943448813 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000b9c0 con 0x7f34e0155140 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f34ddffb700 1 -- 172.21.15.146:0/3943448813 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000bd60 con 0x7f34e0155140 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc738151740 0x7fc73806aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42062/0 (socket says 172.21.15.146:42062) 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 -- 172.21.15.146:0/4162859403 learned_addr learned my addr 172.21.15.146:0/4162859403 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075121281 0 0) 0x7fc73806c2f0 con 0x7fc738151740 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc720003570 con 0x7fc738151740 2024-08-15T16:59:51.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 -- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014fc20 msgr2=0x7f25e0156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f25e5d12700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014fc20 0x7f25e0156200 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f25e0065660 tx=0x7f25d8007820 comp rx=0 tx=0).stop 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 -- 172.21.15.146:0/2586511456 shutdown_connections 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 --2- 172.21.15.146:0/2586511456 >> v2:172.21.15.146:3300/0 conn(0x7f25e00ae5d0 0x7f25e00aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014fc20 0x7f25e0156200 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 --2- 172.21.15.146:0/2586511456 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e014f850 0x7f25e00691f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 -- 172.21.15.146:0/2586511456 >> 172.21.15.146:0/2586511456 conn(0x7f25e0068300 msgr2=0x7f25e00a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f9c1093f700 1 -- 172.21.15.146:0/1003722961 wait complete. 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 Processor -- start 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- start start 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4d59a700 1 -- 172.21.15.146:0/3838399142 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800e070 con 0x7f3a5006df10 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4d59a700 1 -- 172.21.15.146:0/3838399142 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800bbf0 con 0x7f3a5006df10 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7f3a4d59a700 1 -- 172.21.15.146:0/3838399142 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48012430 con 0x7f3a5006df10 2024-08-15T16:59:51.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- 172.21.15.146:0/1635431464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8594155d20 msgr2=0x7f85940b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8594155d20 0x7f85940b0350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f858c009f20 tx=0x7f858c009840 comp rx=0 tx=0).stop 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- 172.21.15.146:0/1635431464 shutdown_connections 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85940aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- 172.21.15.146:0/1635431464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8594155d20 0x7f85940b0350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- 172.21.15.146:0/1635431464 >> 172.21.15.146:0/1635431464 conn(0x7f85940678d0 msgr2=0x7f8594067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- 172.21.15.146:0/1635431464 shutdown_connections 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 0x7ff5ac0aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac1575c0 con 0x7ff5ac0ad430 2024-08-15T16:59:51.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0ad960 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.209+0000 7ff5b0f40700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ac0adc40 con 0x7ff5ac0a9ec0 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34032/0 (socket says 172.21.15.146:34032) 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/683061792 learned_addr learned my addr 172.21.15.146:0/683061792 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5aaffd700 1 --2- 172.21.15.146:0/683061792 >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 0x7ff5ac0aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/683061792 >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 msgr2=0x7ff5ac0aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 --2- 172.21.15.146:0/683061792 >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 0x7ff5ac0aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/683061792 >> v1:172.21.15.146:6789/0 conn(0x7ff5ac0ad430 legacy=0x7ff5ac15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/683061792 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0adf20 con 0x7ff5ac0a9ec0 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5aaffd700 1 --2- 172.21.15.146:0/683061792 >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 0x7ff5ac0aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:51.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc7377fe700 1 --2- 172.21.15.146:0/4162859403 >> v2:172.21.15.146:3300/0 conn(0x7fc73815b240 0x7fc73806a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093100686 0 0) 0x7fc730001490 con 0x7fc738151370 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc73806c2f0 con 0x7fc738151370 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a5006df10 msgr2=0x7f3a5006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a5006df10 0x7f3a5006e390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009e90 tx=0x7f3a48009780 comp rx=0 tx=0).stop 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- 172.21.15.146:0/3838399142 shutdown_connections 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a5006df10 0x7f3a5006e390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 --2- 172.21.15.146:0/3838399142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a500ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- 172.21.15.146:0/3838399142 >> 172.21.15.146:0/3838399142 conn(0x7f3a500678d0 msgr2=0x7f3a50067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5ab7fe700 1 --2- 172.21.15.146:0/683061792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac06b240 tx=0x7ff5a4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1a32f713a7cee2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 -- 172.21.15.146:0/2586511456 shutdown_connections 2024-08-15T16:59:51.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- 172.21.15.146:0/3838399142 shutdown_connections 2024-08-15T16:59:51.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- 172.21.15.146:0/3838399142 wait complete. 2024-08-15T16:59:51.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1794098086 0 0) 0x7fc720003570 con 0x7fc738151740 2024-08-15T16:59:51.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 >> v2:172.21.15.146:3300/0 conn(0x7fc73815b240 msgr2=0x7fc73806a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5aa7fc700 1 -- 172.21.15.146:0/683061792 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a400e070 con 0x7ff5ac0a9ec0 2024-08-15T16:59:51.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5aa7fc700 1 -- 172.21.15.146:0/683061792 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a400ba10 con 0x7ff5ac0a9ec0 2024-08-15T16:59:51.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5aa7fc700 1 -- 172.21.15.146:0/683061792 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40092e0 con 0x7ff5ac0a9ec0 2024-08-15T16:59:51.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c0b1950 0x7f9c0c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c13b630 0x7f9c0c135cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 0x7f9c0c1361f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c1367e0 con 0x7f9c0c13bf10 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c1766c0 con 0x7f9c0c13b630 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0c1769a0 con 0x7f9c0c0b1950 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 0x7f9c0c1361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 0x7f9c0c1361f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49684/0 (socket says 172.21.15.146:49684) 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/538351707 learned_addr learned my addr 172.21.15.146:0/538351707 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0a59c700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c0b1950 0x7f9c0c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- 172.21.15.146:0/1635431464 wait complete. 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 Processor -- start 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 --2- 172.21.15.146:0/4162859403 >> v2:172.21.15.146:3300/0 conn(0x7fc73815b240 0x7fc73806a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 >> v1:172.21.15.146:6789/0 conn(0x7fc738151370 legacy=0x7fc73815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73806b6e0 con 0x7fc738151740 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7fc728001470 con 0x7fc738151740 2024-08-15T16:59:51.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 >> v1:172.21.15.161:6789/0 conn(0x7fc738151740 legacy=0x7fc73806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c0b1950 msgr2=0x7f9c0c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c0c0b1950 0x7f9c0c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c13b630 msgr2=0x7f9c0c135cb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c0c13b630 0x7f9c0c135cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c00009420 con 0x7f9c0c13bf10 2024-08-15T16:59:51.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- start start 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc720003bf0 0x7fc720003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc720004810 0x7fc720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 0x7fc72000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73806c780 con 0x7fc720003bf0 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72000c080 con 0x7fc720004810 2024-08-15T16:59:51.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc72000c240 con 0x7fc7200051c0 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 -- 172.21.15.146:0/2586511456 wait complete. 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 -- 172.21.15.146:0/683061792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac0a9ec0 msgr2=0x7ff5ac0acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/683061792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac06b240 tx=0x7ff5a4009840 comp rx=0 tx=0).stop 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 -- 172.21.15.146:0/683061792 shutdown_connections 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/683061792 >> v2:172.21.15.146:3300/0 conn(0x7ff5ac0a91c0 0x7ff5ac0aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/683061792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac0a9ec0 0x7ff5ac0acef0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac06b240 tx=0x7ff5a4009840 comp rx=0 tx=0).stop 2024-08-15T16:59:51.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 -- 172.21.15.146:0/683061792 >> 172.21.15.146:0/683061792 conn(0x7ff5ac066e00 msgr2=0x7ff5ac067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 Processor -- start 2024-08-15T16:59:51.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c0ad9d700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 0x7f9c0c1361f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c790 tx=0x7f9bfc007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 Processor -- start 2024-08-15T16:59:51.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc7377fe700 1 --2- 172.21.15.146:0/4162859403 >> v2:172.21.15.146:3300/0 conn(0x7fc73815b240 0x7fc73806a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:51.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 -- 172.21.15.146:0/683061792 shutdown_connections 2024-08-15T16:59:51.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 0x7fc72000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc005c00 con 0x7f9c0c13bf10 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc005d60 con 0x7f9c0c13bf10 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0c176c80 con 0x7f9c0c13bf10 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0072e0 con 0x7f9c0c13bf10 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0c176330 con 0x7f9c0c13bf10 2024-08-15T16:59:51.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc7377fe700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc720004810 0x7fc720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f859406df10 0x7f8594156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85941673e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8594155d20 0x7f8594166110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594167920 con 0x7f859406df10 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594157590 con 0x7f8594155d20 2024-08-15T16:59:51.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f85990c7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859414d450 con 0x7f85940ad7c0 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 -- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc720004810 msgr2=0x7fc720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc720004810 0x7fc720004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 -- 172.21.15.146:0/4162859403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc720003bf0 msgr2=0x7fc720003fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc720003bf0 0x7fc720003fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc73cce6700 1 -- 172.21.15.146:0/4162859403 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73806b6e0 con 0x7fc7200051c0 2024-08-15T16:59:51.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73cce6700 1 -- 172.21.15.146:0/4162859403 reap_dead start 2024-08-15T16:59:51.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85988c6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8594155d20 0x7f8594166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85988c6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8594155d20 0x7f8594166110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34062/0 (socket says 172.21.15.146:34062) 2024-08-15T16:59:51.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85988c6700 1 -- 172.21.15.146:0/1085742552 learned_addr learned my addr 172.21.15.146:0/1085742552 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85941673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73cce6700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 0x7fc72000ba60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c460 tx=0x7fc7280079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe284068109cf53 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f25e5d12700 1 -- start start 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e00ae5d0 0x7f25e00ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014f850 0x7f25e00a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 0x7f25e00aa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00b1aa0 con 0x7f25e014fc20 2024-08-15T16:59:51.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0152630 con 0x7f25e014f850 2024-08-15T16:59:51.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25e5d12700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00a99c0 con 0x7f25e00ae5d0 2024-08-15T16:59:51.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 -- 172.21.15.146:0/3943448813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e0155140 msgr2=0x7f34e0155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 --2- 172.21.15.146:0/3943448813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e0155140 0x7f34e0155510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34e0068ba0 tx=0x7f34d0009250 comp rx=0 tx=0).stop 2024-08-15T16:59:51.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72800cd10 con 0x7fc7200051c0 2024-08-15T16:59:51.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25df7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e00ae5d0 0x7f25e00ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 0x7f25e00aa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 0x7f25e00aa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49698/0 (socket says 172.21.15.146:49698) 2024-08-15T16:59:51.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 -- 172.21.15.146:0/469604911 learned_addr learned my addr 172.21.15.146:0/469604911 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc392ce5700 1 Processor -- start 2024-08-15T16:59:51.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7fc392ce5700 1 -- start start 2024-08-15T16:59:51.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72800ce90 con 0x7fc7200051c0 2024-08-15T16:59:51.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc736ffd700 1 -- 172.21.15.146:0/4162859403 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728005b70 con 0x7fc7200051c0 2024-08-15T16:59:51.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7ff5b0f40700 1 -- 172.21.15.146:0/683061792 wait complete. 2024-08-15T16:59:51.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 Processor -- start 2024-08-15T16:59:51.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.213+0000 7f3a4f59e700 1 -- start start 2024-08-15T16:59:51.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 0x7f3a5016d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a501462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a50155d20 0x7f3a500ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 0x7f3a5016d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 0x7f3a5016d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49702/0 (socket says 172.21.15.146:49702) 2024-08-15T16:59:51.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc392ce5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc38c0b0a90 0x7fc38c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 -- 172.21.15.146:0/2345890857 learned_addr learned my addr 172.21.15.146:0/2345890857 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc392ce5700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc38c06e450 0x7fc38c15efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc392ce5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c06e850 con 0x7fc38c157a10 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc392ce5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0aed20 con 0x7fc38c0b0a90 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc392ce5700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0b1020 con 0x7fc38c06e450 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9bf0000fd0 con 0x7f9c0c13bf10 2024-08-15T16:59:51.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a50155d20 0x7f3a500ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4dd9b700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a501462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc38c0b0a90 0x7fc38c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50157420 con 0x7f3a5006df10 2024-08-15T16:59:51.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50156af0 con 0x7f3a50155d20 2024-08-15T16:59:51.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc38c0b0a90 0x7fc38c06df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49718/0 (socket says 172.21.15.146:49718) 2024-08-15T16:59:51.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 -- 172.21.15.146:0/2684911355 learned_addr learned my addr 172.21.15.146:0/2684911355 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3924e4700 1 --2- 172.21.15.146:0/2684911355 >> v2:172.21.15.161:3300/0 conn(0x7fc38c06e450 0x7fc38c15efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50156dd0 con 0x7f3a500b0350 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73d4e7700 1 -- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 msgr2=0x7fc72000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73d4e7700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 0x7fc72000ba60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c460 tx=0x7fc7280079d0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73d4e7700 1 -- 172.21.15.146:0/4162859403 shutdown_connections 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73d4e7700 1 --2- 172.21.15.146:0/4162859403 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc7200051c0 0x7fc72000ba60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc73d4e7700 1 -- 172.21.15.146:0/4162859403 >> 172.21.15.146:0/4162859403 conn(0x7fc73806cc40 msgr2=0x7fc73806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 -- 172.21.15.146:0/3943448813 shutdown_connections 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 --2- 172.21.15.146:0/3943448813 >> v2:172.21.15.146:3300/0 conn(0x7f34e006b390 0x7f34e00aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 --2- 172.21.15.146:0/3943448813 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e0155140 0x7f34e0155510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 -- 172.21.15.146:0/3943448813 >> 172.21.15.146:0/3943448813 conn(0x7f34e0067810 msgr2=0x7f34e0067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 -- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e00ae5d0 msgr2=0x7f25e00ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25e00ae5d0 0x7f25e00ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 -- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014f850 msgr2=0x7f25e00a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25e014f850 0x7f25e00a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f25dffff700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d800c050 con 0x7f25e014fc20 2024-08-15T16:59:51.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 -- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8594155d20 msgr2=0x7f8594166110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8594155d20 0x7f8594166110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 -- 172.21.15.146:0/1085742552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f859406df10 msgr2=0x7f8594156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f859406df10 0x7f8594156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c009ae0 con 0x7f85940ad7c0 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85937fe700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85941673e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f858800ea50 tx=0x7f85880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800c7f0 con 0x7f85940ad7c0 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858800c950 con 0x7f85940ad7c0 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800cd10 con 0x7f85940ad7c0 2024-08-15T16:59:51.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8594156220 con 0x7f85940ad7c0 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 -- start start 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 0x7ff5ac156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac0ad430 0x7ff5ac159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac15a0a0 0x7ff5ac16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac15a910 con 0x7ff5ac0ad430 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac15abf0 con 0x7ff5ac15a0a0 2024-08-15T16:59:51.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5b0f40700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac15aed0 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc390ce1700 1 -- 172.21.15.146:0/2684911355 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260316408 0 0) 0x7fc38c06e850 con 0x7fc38c157a10 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 -- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 msgr2=0x7f3a501462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a500b0350 0x7f3a501462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 -- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a50155d20 msgr2=0x7f3a500ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a50155d20 0x7f3a500ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a48009a20 con 0x7f3a5006df10 2024-08-15T16:59:51.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc390ce1700 1 -- 172.21.15.146:0/2684911355 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc378003570 con 0x7fc38c157a10 2024-08-15T16:59:51.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 -- 172.21.15.146:0/3943448813 shutdown_connections 2024-08-15T16:59:51.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f34e5736700 1 -- 172.21.15.146:0/3943448813 wait complete. 2024-08-15T16:59:51.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 -- 172.21.15.146:0/2684911355 >> v2:172.21.15.161:3300/0 conn(0x7fc38c06e450 msgr2=0x7fc38c15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 --2- 172.21.15.146:0/2684911355 >> v2:172.21.15.161:3300/0 conn(0x7fc38c06e450 0x7fc38c15efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 -- 172.21.15.146:0/2684911355 >> v1:172.21.15.146:6789/0 conn(0x7fc38c157a10 legacy=0x7fc38c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 -- 172.21.15.146:0/2684911355 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0b1300 con 0x7fc38c0b0a90 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 0x7ff5ac156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 0x7ff5ac156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59612/0 (socket says 172.21.15.146:59612) 2024-08-15T16:59:51.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/1337514824 learned_addr learned my addr 172.21.15.146:0/1337514824 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc3914e2700 1 --2- 172.21.15.146:0/2684911355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c06df10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc38c1470b0 tx=0x7fc384009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c3d9704ae5f2f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85941567e0 con 0x7f85940ad7c0 2024-08-15T16:59:51.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc390ce1700 1 -- 172.21.15.146:0/2684911355 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38400e070 con 0x7fc38c0b0a90 2024-08-15T16:59:51.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac15a0a0 msgr2=0x7ff5ac16a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5abfff700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac15a0a0 0x7ff5ac16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9bfc01b3b0 con 0x7f9c0c13bf10 2024-08-15T16:59:51.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5ac15a0a0 0x7ff5ac16a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/1337514824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac0ad430 msgr2=0x7ff5ac159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff5ac0ad430 0x7ff5ac159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a4009ae0 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f9bfa7fc700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9bec060930 0x7f9bec062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7ff5ab7fe700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 0x7ff5ac156870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4003000 tx=0x7ff5a4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85940643a0 con 0x7f85940ad7c0 2024-08-15T16:59:51.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9bfc07ca00 con 0x7f9c0c13bf10 2024-08-15T16:59:51.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9bfa7fc700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7f9bec068b40 0x7f9bec06af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9bec06b600 con 0x7f9bec068b40 2024-08-15T16:59:51.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc390ce1700 1 -- 172.21.15.146:0/2684911355 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38400b970 con 0x7fc38c0b0a90 2024-08-15T16:59:51.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7fc390ce1700 1 -- 172.21.15.146:0/2684911355 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3840092e0 con 0x7fc38c0b0a90 2024-08-15T16:59:51.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bfc07cdd0 con 0x7f9c0c13bf10 2024-08-15T16:59:51.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9c0a59c700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9bec060930 0x7f9bec062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9c09d9b700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7f9bec068b40 0x7f9bec06af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4012a10 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a4012b90 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a401d420 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 Processor -- start 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac15b1b0 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac15a540 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a4e59c700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 0x7f3a5016d930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000c560 tx=0x7f3a40001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40005b30 con 0x7f3a5006df10 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.217+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40005c90 con 0x7f3a5006df10 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40007320 con 0x7f3a5006df10 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a501570b0 con 0x7f3a5006df10 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a500cb290 con 0x7f3a5006df10 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 -- 172.21.15.146:0/2684911355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 msgr2=0x7fc38c06df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2684911355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c06df10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc38c1470b0 tx=0x7fc384009840 comp rx=0 tx=0).stop 2024-08-15T16:59:51.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc73d4e7700 1 -- 172.21.15.146:0/4162859403 shutdown_connections 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8588013680 con 0x7f85940ad7c0 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f85800608e0 0x7f8580062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f858807cd40 con 0x7f85940ad7c0 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7f8580068a70 0x7f858006ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f858006b530 con 0x7f8580068a70 2024-08-15T16:59:51.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8588081030 con 0x7f85940ad7c0 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f8593fff700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f85800608e0 0x7f8580062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 -- 172.21.15.146:0/2684911355 shutdown_connections 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2684911355 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c06df10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2684911355 >> v2:172.21.15.161:3300/0 conn(0x7fc38c06e450 0x7fc38c15efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc392ce5700 1 -- 172.21.15.146:0/2684911355 >> 172.21.15.146:0/2684911355 conn(0x7fc38c0678d0 msgr2=0x7fc38c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5ac0643a0 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f85988c6700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7f8580068a70 0x7f858006ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 -- start start 2024-08-15T16:59:51.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25dffff700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 0x7f25e00aa530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e870 tx=0x7f25d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9c09d9b700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7f9bec068b40 0x7f9bec06af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a500643a0 con 0x7f3a5006df10 2024-08-15T16:59:51.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f8593fff700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f85800608e0 0x7f8580062d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f858c009880 tx=0x7f858c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f85988c6700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7f8580068a70 0x7f858006ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff5a401d810 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9c0a59c700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9bec060930 0x7f9bec062de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009fd0 tx=0x7f9c0400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 0x7f34e01609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e006e2b0 0x7f34e01716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34e0155140 0x7f34e0171c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0157420 con 0x7f34e006b390 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e016f9f0 con 0x7f34e006e2b0 2024-08-15T16:59:51.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34e5736700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0160f10 con 0x7f34e0155140 2024-08-15T16:59:51.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== osd.3 v2:172.21.15.146:6800/24249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9bec06b600 con 0x7f9bec068b40 2024-08-15T16:59:51.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400c430 con 0x7f25e014fc20 2024-08-15T16:59:51.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7fc73d4e7700 1 -- 172.21.15.146:0/4162859403 wait complete. 2024-08-15T16:59:51.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d400c590 con 0x7f25e014fc20 2024-08-15T16:59:51.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d400c910 con 0x7f25e014fc20 2024-08-15T16:59:51.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff598060860 0x7ff598062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5a407e830 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7ff5980689f0 0x7ff59806ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff59806b4b0 con 0x7ff5980689f0 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5a407ec00 con 0x7ff5ac0a91c0 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7ff5aaffd700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff598060860 0x7ff598062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00a9ca0 con 0x7f25e014fc20 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0151ee0 con 0x7f25e014fc20 2024-08-15T16:59:51.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3a4001b3b0 con 0x7f3a5006df10 2024-08-15T16:59:51.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 0x7f34e01609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 0x7f34e01609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49728/0 (socket says 172.21.15.146:49728) 2024-08-15T16:59:51.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 -- 172.21.15.146:0/534588659 learned_addr learned my addr 172.21.15.146:0/534588659 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34df7fe700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34e0155140 0x7f34e0171c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 -- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34e0155140 msgr2=0x7f34e0171c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f34e0155140 0x7f34e0171c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 -- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e006e2b0 msgr2=0x7f34e01716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f34e006e2b0 0x7f34e01716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 -- 172.21.15.146:0/534588659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00094f0 con 0x7f34e006b390 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f34deffd700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 0x7f34e01609d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d00090b0 tx=0x7f34d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.221+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== osd.6 v2:172.21.15.161:6800/24462 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f858006b530 con 0x7f8580068a70 2024-08-15T16:59:51.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7ff5abfff700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7ff5980689f0 0x7ff59806ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a3e7fc700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a300608b0 0x7f3a30062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.294 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a4007c620 con 0x7f3a5006df10 2024-08-15T16:59:51.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7ff5aaffd700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff598060860 0x7ff598062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac063c10 tx=0x7ff5a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a4dd9b700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a300608b0 0x7f3a30062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a3e7fc700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f3a30068950 0x7f3a3006ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a3006b410 con 0x7f3a30068950 2024-08-15T16:59:51.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a4007c9f0 con 0x7f3a5006df10 2024-08-15T16:59:51.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7fc73d4e7700 1 Processor -- start 2024-08-15T16:59:51.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25e0154a20 con 0x7f25e014fc20 2024-08-15T16:59:51.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7ff5abfff700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7ff5980689f0 0x7ff59806ae00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f3a4dd9b700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a300608b0 0x7f3a30062d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006c600 tx=0x7f3a440091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f25d400cd40 con 0x7f25e014fc20 2024-08-15T16:59:51.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f25cc060960 0x7f25cc062e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25d407c920 con 0x7f25e014fc20 2024-08-15T16:59:51.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7f25cc068af0 0x7f25cc06af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25df7fe700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f25cc060960 0x7f25cc062e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25cc06b5b0 con 0x7f25cc068af0 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25deffd700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7f25cc068af0 0x7f25cc06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d404cdf0 con 0x7f25e014fc20 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== osd.7 v2:172.21.15.161:6808/24463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff59806b4b0 con 0x7ff5980689f0 2024-08-15T16:59:51.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25deffd700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7f25cc068af0 0x7f25cc06af00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25df7fe700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f25cc060960 0x7f25cc062e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009d40 tx=0x7f25d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== osd.2 v2:172.21.15.146:6816/24250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f25cc06b5b0 con 0x7f25cc068af0 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- start start 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 0x7fc7380687a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7381699a0 0x7fc7380c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc73814d490 0x7fc73814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73814de50 con 0x7fc7200051c0 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738069b70 con 0x7fc7381699a0 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc738068ce0 con 0x7fc73814d490 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 0x7fc7380687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 0x7fc7380687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49730/0 (socket says 172.21.15.146:49730) 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 -- 172.21.15.146:0/359062304 learned_addr learned my addr 172.21.15.146:0/359062304 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73cce6700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc73814d490 0x7fc73814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc7377fe700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7381699a0 0x7fc7380c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 -- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc73814d490 msgr2=0x7fc73814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc73814d490 0x7fc73814d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 -- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7381699a0 msgr2=0x7fc7380c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7381699a0 0x7fc7380c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 -- 172.21.15.146:0/359062304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738068f60 con 0x7fc7200051c0 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc7377fe700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7381699a0 0x7fc7380c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc737fff700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 0x7fc7380687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7300031b0 tx=0x7fc7300027f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000cea0 con 0x7fc7200051c0 2024-08-15T16:59:51.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730015b70 con 0x7fc7200051c0 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- 172.21.15.146:0/359062304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc738069240 con 0x7fc7200051c0 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000a590 con 0x7fc7200051c0 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc73d4e7700 1 -- 172.21.15.146:0/359062304 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc738069800 con 0x7fc7200051c0 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc73000a940 con 0x7fc7200051c0 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0006bc0 con 0x7f34e006b390 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d001a450 con 0x7f34e006b390 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d001a810 con 0x7f34e006b390 2024-08-15T16:59:51.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e0157140 con 0x7f34e006b390 2024-08-15T16:59:51.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- 172.21.15.146:0/2684911355 shutdown_connections 2024-08-15T16:59:51.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e016f680 con 0x7f34e006b390 2024-08-15T16:59:51.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f34d001ab80 con 0x7f34e006b390 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f34cc060860 0x7f34cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34d007d640 con 0x7f34e006b390 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34de7fc700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f34cc060860 0x7f34cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34e5736700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f34e0156430 0x7f34e0156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 --> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34e00645c0 con 0x7f34e0156430 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34df7fe700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f34e0156430 0x7f34e0156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34df7fe700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f34e0156430 0x7f34e0156800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.225+0000 7f34de7fc700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f34cc060860 0x7f34cc062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34e00aea10 tx=0x7f34d4006d10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.229+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== osd.0 v2:172.21.15.146:6824/24267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f34e00645c0 con 0x7f34e0156430 2024-08-15T16:59:51.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- 172.21.15.146:0/2684911355 wait complete. 2024-08-15T16:59:51.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f3a30068950 0x7f3a3006ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7f3a4ed9d700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f3a30068950 0x7f3a3006ad60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== osd.4 v2:172.21.15.161:6824/24465 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f3a3006b410 con 0x7f3a30068950 2024-08-15T16:59:51.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc71ffff700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc724060860 0x7fc724062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc730084b90 con 0x7fc7200051c0 2024-08-15T16:59:51.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc7377fe700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc724060860 0x7fc724062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc7377fe700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc724060860 0x7fc724062d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc738066b10 tx=0x7fc72c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc71dffb700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7fc73805e330 0x7fc73805e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 --> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7380638f0 con 0x7fc73805e330 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc73cce6700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7fc73805e330 0x7fc73805e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc73cce6700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7fc73805e330 0x7fc73805e700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== osd.1 v2:172.21.15.146:6802/24248 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc7380638f0 con 0x7fc73805e330 2024-08-15T16:59:51.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 Processor -- start 2024-08-15T16:59:51.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- start start 2024-08-15T16:59:51.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc38c06e450 0x7fc38c13ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc38c157a10 0x7fc38c144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c16dea0 con 0x7fc38c0b0a90 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c142480 con 0x7fc38c157a10 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc392ce5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c172350 con 0x7fc38c06e450 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49744/0 (socket says 172.21.15.146:49744) 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 -- 172.21.15.146:0/2417633671 learned_addr learned my addr 172.21.15.146:0/2417633671 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc391ce3700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc38c06e450 0x7fc38c13ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3924e4700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc38c157a10 0x7fc38c144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 -- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc38c06e450 msgr2=0x7fc38c13ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc38c06e450 0x7fc38c13ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 -- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc38c157a10 msgr2=0x7fc38c144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc38c157a10 0x7fc38c144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.233+0000 7fc3914e2700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384009ae0 con 0x7fc38c0b0a90 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.237+0000 7fc3914e2700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c144300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06beb0 tx=0x7fc384018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384008870 con 0x7fc38c0b0a90 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3840089d0 con 0x7fc38c0b0a90 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384008d90 con 0x7fc38c0b0a90 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c141b50 con 0x7fc38c0b0a90 2024-08-15T16:59:51.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.241+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c142110 con 0x7fc38c0b0a90 2024-08-15T16:59:51.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 --> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5ac0643a0 con 0x7ff5980689f0 2024-08-15T16:59:51.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc36c000fd0 con 0x7fc38c0b0a90 2024-08-15T16:59:51.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 --> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25e00645c0 con 0x7f25cc068af0 2024-08-15T16:59:51.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc384003040 con 0x7fc38c0b0a90 2024-08-15T16:59:51.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5937fe700 1 -- 172.21.15.146:0/1337514824 <== osd.7 v2:172.21.15.161:6808/24463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5ac0643a0 con 0x7ff5980689f0 2024-08-15T16:59:51.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc378060d10 0x7fc3780631c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25c77fe700 1 -- 172.21.15.146:0/469604911 <== osd.2 v2:172.21.15.146:6816/24250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25e00645c0 con 0x7f25cc068af0 2024-08-15T16:59:51.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7ff5980689f0 msgr2=0x7ff59806ae00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc38407eb60 con 0x7fc38c0b0a90 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 --> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8594153370 con 0x7f8580068a70 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7ff5980689f0 0x7ff59806ae00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7fc378068db0 0x7fc37806b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc37806b870 con 0x7fc378068db0 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38407ef30 con 0x7fc38c0b0a90 2024-08-15T16:59:51.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff598060860 msgr2=0x7ff598062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff598060860 0x7ff598062d10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac063c10 tx=0x7ff5a0008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc391ce3700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc378060d10 0x7fc3780631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 msgr2=0x7ff5ac156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 --2- 172.21.15.146:0/1337514824 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5ac0a91c0 0x7ff5ac156870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4003000 tx=0x7ff5a4017000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5abfff700 1 -- 172.21.15.146:0/1337514824 reap_dead start 2024-08-15T16:59:51.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 shutdown_connections 2024-08-15T16:59:51.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 >> 172.21.15.146:0/1337514824 conn(0x7ff5ac066e00 msgr2=0x7ff5ac0aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 shutdown_connections 2024-08-15T16:59:51.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7ff5b0f40700 1 -- 172.21.15.146:0/1337514824 wait complete. 2024-08-15T16:59:51.318 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc3924e4700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7fc378068db0 0x7fc37806b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 --> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9bf0002d30 con 0x7f9bec068b40 2024-08-15T16:59:51.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f857bfff700 1 -- 172.21.15.146:0/1085742552 <== osd.6 v2:172.21.15.161:6800/24462 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8594153370 con 0x7f8580068a70 2024-08-15T16:59:51.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc391ce3700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc378060d10 0x7fc3780631c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc37c007fd0 tx=0x7fc37c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9bfa7fc700 1 -- 172.21.15.146:0/538351707 <== osd.3 v2:172.21.15.146:6800/24249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9bf0002d30 con 0x7f9bec068b40 2024-08-15T16:59:51.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7f25cc068af0 msgr2=0x7f25cc06af00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7f25cc068af0 0x7f25cc06af00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7fc3924e4700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7fc378068db0 0x7fc37806b1c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7f9bec068b40 msgr2=0x7f9bec06af50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7f9bec068b40 0x7f9bec06af50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7f8580068a70 msgr2=0x7f858006ae80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.321 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f85990c7700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7f8580068a70 0x7f858006ae80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9bec060930 msgr2=0x7f9bec062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9bec060930 0x7f9bec062de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c04009fd0 tx=0x7f9c0400b040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 msgr2=0x7f9c0c1361f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 --2- 172.21.15.146:0/538351707 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9c0c13bf10 0x7f9c0c1361f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00c790 tx=0x7f9bfc007d90 comp rx=0 tx=0).stop 2024-08-15T16:59:51.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c0ad9d700 1 -- 172.21.15.146:0/538351707 reap_dead start 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f25cc060960 msgr2=0x7f25cc062e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f25cc060960 0x7f25cc062e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009d40 tx=0x7f25d0008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 msgr2=0x7f25e00aa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25e5d12700 1 --2- 172.21.15.146:0/469604911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e014fc20 0x7f25e00aa530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e870 tx=0x7f25d4008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 shutdown_connections 2024-08-15T16:59:51.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 >> 172.21.15.146:0/538351707 conn(0x7f9c0c0678d0 msgr2=0x7f9c0c069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f25dffff700 1 -- 172.21.15.146:0/469604911 reap_dead start 2024-08-15T16:59:51.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f85800608e0 msgr2=0x7f8580062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.249+0000 7f85990c7700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f85800608e0 0x7f8580062d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f858c009880 tx=0x7f858c017000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 shutdown_connections 2024-08-15T16:59:51.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 >> 172.21.15.146:0/469604911 conn(0x7f25e0068300 msgr2=0x7f25e0156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 msgr2=0x7f85941673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 shutdown_connections 2024-08-15T16:59:51.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 shutdown_connections 2024-08-15T16:59:51.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f9c1093f700 1 -- 172.21.15.146:0/538351707 wait complete. 2024-08-15T16:59:51.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f25e5d12700 1 -- 172.21.15.146:0/469604911 wait complete. 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 --2- 172.21.15.146:0/1085742552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85940ad7c0 0x7f85941673e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f858800ea50 tx=0x7f85880091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85988c6700 1 -- 172.21.15.146:0/1085742552 reap_dead start 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 shutdown_connections 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 >> 172.21.15.146:0/1085742552 conn(0x7f85940678d0 msgr2=0x7f8594151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 shutdown_connections 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7f85990c7700 1 -- 172.21.15.146:0/1085742552 wait complete. 2024-08-15T16:59:51.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.253+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== osd.5 v2:172.21.15.161:6816/24464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc37806b870 con 0x7fc378068db0 2024-08-15T16:59:51.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.261+0000 7f3a4f59e700 1 -- 172.21.15.146:0/2345890857 --> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a5006f5f0 con 0x7f3a30068950 2024-08-15T16:59:51.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.261+0000 7f3a3e7fc700 1 -- 172.21.15.146:0/2345890857 <== osd.4 v2:172.21.15.161:6824/24465 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a5006f5f0 con 0x7f3a30068950 2024-08-15T16:59:51.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f3a30068950 msgr2=0x7f3a3006ad60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f3a30068950 0x7f3a3006ad60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.329 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.330 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-15T16:59:51.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a300608b0 msgr2=0x7f3a30062d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a300608b0 0x7f3a30062d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a5006c600 tx=0x7f3a440091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 msgr2=0x7f3a5016d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 --2- 172.21.15.146:0/2345890857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a5006df10 0x7f3a5016d930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a4000c560 tx=0x7f3a40001020 comp rx=0 tx=0).stop 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a4ed9d700 1 -- 172.21.15.146:0/2345890857 reap_dead start 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 shutdown_connections 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 >> 172.21.15.146:0/2345890857 conn(0x7f3a500678d0 msgr2=0x7f3a501485c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 shutdown_connections 2024-08-15T16:59:51.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7f3a2ffff700 1 -- 172.21.15.146:0/2345890857 wait complete. 2024-08-15T16:59:51.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 --> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7380cba00 con 0x7fc73805e330 2024-08-15T16:59:51.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71ffff700 1 -- 172.21.15.146:0/359062304 <== osd.1 v2:172.21.15.146:6802/24248 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7380cba00 con 0x7fc73805e330 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7fc73805e330 msgr2=0x7fc73805e700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7fc73805e330 0x7fc73805e700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc724060860 msgr2=0x7fc724062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc724060860 0x7fc724062d10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc738066b10 tx=0x7fc72c008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 msgr2=0x7fc7380687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 --2- 172.21.15.146:0/359062304 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7200051c0 0x7fc7380687a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7300031b0 tx=0x7fc7300027f0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc73cce6700 1 -- 172.21.15.146:0/359062304 reap_dead start 2024-08-15T16:59:51.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 shutdown_connections 2024-08-15T16:59:51.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 >> 172.21.15.146:0/359062304 conn(0x7fc73806cc40 msgr2=0x7fc738155130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 shutdown_connections 2024-08-15T16:59:51.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.265+0000 7fc71dffb700 1 -- 172.21.15.146:0/359062304 wait complete. 2024-08-15T16:59:51.335 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.336 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-15T16:59:51.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.269+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 --> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34e0154f20 con 0x7f34e0156430 2024-08-15T16:59:51.337 INFO:teuthology.orchestra.run.smithi146.stdout:60129542151 2024-08-15T16:59:51.337 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-15T16:59:51.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.269+0000 7f34c6ffd700 1 -- 172.21.15.146:0/534588659 <== osd.0 v2:172.21.15.146:6824/24267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f34e0154f20 con 0x7f34e0156430 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f34e0156430 msgr2=0x7f34e0156800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f34e0156430 0x7f34e0156800 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f34cc060860 msgr2=0x7f34cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f34cc060860 0x7f34cc062d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f34e00aea10 tx=0x7f34d4006d10 comp rx=0 tx=0).stop 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 msgr2=0x7f34e01609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 --2- 172.21.15.146:0/534588659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34e006b390 0x7f34e01609d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f34d00090b0 tx=0x7f34d0017000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34df7fe700 1 -- 172.21.15.146:0/534588659 reap_dead start 2024-08-15T16:59:51.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 shutdown_connections 2024-08-15T16:59:51.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 >> 172.21.15.146:0/534588659 conn(0x7f34e0067810 msgr2=0x7f34e00b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 shutdown_connections 2024-08-15T16:59:51.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7f34e5736700 1 -- 172.21.15.146:0/534588659 wait complete. 2024-08-15T16:59:51.341 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.341 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-15T16:59:51.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 --> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc36c002c90 con 0x7fc378068db0 2024-08-15T16:59:51.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc381ffb700 1 -- 172.21.15.146:0/2417633671 <== osd.5 v2:172.21.15.161:6816/24464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc36c002c90 con 0x7fc378068db0 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7fc378068db0 msgr2=0x7fc37806b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7fc378068db0 0x7fc37806b1c0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc378060d10 msgr2=0x7fc3780631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc378060d10 0x7fc3780631c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc37c007fd0 tx=0x7fc37c007d50 comp rx=0 tx=0).stop 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 msgr2=0x7fc38c144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 --2- 172.21.15.146:0/2417633671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc38c0b0a90 0x7fc38c144300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc38c06beb0 tx=0x7fc384018000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc3924e4700 1 -- 172.21.15.146:0/2417633671 reap_dead start 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 shutdown_connections 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 >> 172.21.15.146:0/2417633671 conn(0x7fc38c0678d0 msgr2=0x7fc38c06a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 shutdown_connections 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.273+0000 7fc392ce5700 1 -- 172.21.15.146:0/2417633671 wait complete. 2024-08-15T16:59:51.344 INFO:teuthology.orchestra.run.smithi146.stdout:60129542149 2024-08-15T16:59:51.345 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-15T16:59:51.346 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.346 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-15T16:59:51.347 INFO:teuthology.orchestra.run.smithi146.stdout:60129542151 2024-08-15T16:59:51.347 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-15T16:59:51.348 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.349 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-15T16:59:51.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 Processor -- start 2024-08-15T16:59:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 -- start start 2024-08-15T16:59:51.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 0x7f05c80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c813cf90 con 0x7f05c813bf10 2024-08-15T16:59:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c813c380 con 0x7f05c80b1950 2024-08-15T16:59:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cf0a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c813c660 con 0x7f05c813b630 2024-08-15T16:59:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05c7fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f05c813bf10 0x7f05c806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60094/0 (socket says 172.21.15.146:60094) 2024-08-15T16:59:51.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05c7fff700 1 -- 172.21.15.146:0/482007104 learned_addr learned my addr 172.21.15.146:0/482007104 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607855780 0 0) 0x7f05c813cf90 con 0x7f05c813bf10 2024-08-15T16:59:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cce43700 1 --2- 172.21.15.146:0/482007104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 0x7f05c80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b0003570 con 0x7f05c813bf10 2024-08-15T16:59:51.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217519696 0 0) 0x7f05c813c380 con 0x7f05c80b1950 2024-08-15T16:59:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cce43700 1 -- 172.21.15.146:0/482007104 >> v1:172.21.15.161:6789/0 conn(0x7f05c80b1950 legacy=0x7f05c813d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cce43700 1 -- 172.21.15.146:0/482007104 >> v1:172.21.15.146:6789/0 conn(0x7f05c813bf10 legacy=0x7f05c806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cce43700 1 -- 172.21.15.146:0/482007104 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c813c940 con 0x7f05c813b630 2024-08-15T16:59:51.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f55fce72700 1 Processor -- start 2024-08-15T16:59:51.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f55fce72700 1 -- start start 2024-08-15T16:59:51.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.509+0000 7f05cce43700 1 --2- 172.21.15.146:0/482007104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 0x7f05c80b1410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05b80092a0 tx=0x7f05b8009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7366184f19f53273 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc001470 con 0x7f05c813b630 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b8009730 con 0x7f05c813b630 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05c77fe700 1 -- 172.21.15.146:0/482007104 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b800c870 con 0x7f05c813b630 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- 172.21.15.146:0/482007104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 msgr2=0x7f05c80b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 --2- 172.21.15.146:0/482007104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 0x7f05c80b1410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05b80092a0 tx=0x7f05b8009a30 comp rx=0 tx=0).stop 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- 172.21.15.146:0/482007104 shutdown_connections 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 --2- 172.21.15.146:0/482007104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c813b630 0x7f05c80b1410 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f05b80092a0 tx=0x7f05b8009a30 comp rx=0 tx=0).stop 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- 172.21.15.146:0/482007104 >> 172.21.15.146:0/482007104 conn(0x7f05c80678d0 msgr2=0x7f05c8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- 172.21.15.146:0/482007104 shutdown_connections 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55f806e680 0x7f55f806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 0x7f55f8159940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f55f8159e80 0x7f55f80ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f815a8e0 con 0x7f55f8159e80 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f815a600 con 0x7f55f806e680 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80ab9b0 con 0x7f55f80ac490 2024-08-15T16:59:51.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- 172.21.15.146:0/482007104 wait complete. 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f55f8159e80 0x7f55f80ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f5d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 0x7f55f8159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f5d9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 0x7f55f8159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59636/0 (socket says 172.21.15.146:59636) 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f55f8159e80 0x7f55f80ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49748/0 (socket says 172.21.15.146:49748) 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f5d9b700 1 -- 172.21.15.146:0/1439632535 learned_addr learned my addr 172.21.15.146:0/1439632535 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 Processor -- start 2024-08-15T16:59:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f659c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f55f806e680 0x7f55f806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 -- 172.21.15.146:0/1439632535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 msgr2=0x7f55f8159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 --2- 172.21.15.146:0/1439632535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 0x7f55f8159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 -- 172.21.15.146:0/1439632535 >> v2:172.21.15.161:3300/0 conn(0x7f55f806e680 msgr2=0x7f55f806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 --2- 172.21.15.146:0/1439632535 >> v2:172.21.15.161:3300/0 conn(0x7f55f806e680 0x7f55f806ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 -- 172.21.15.146:0/1439632535 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80abc90 con 0x7f55f8159e80 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f6d9d700 1 --2- 172.21.15.146:0/1439632535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80ade10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55f806fbc0 tx=0x7f55ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307ceba270b0f982 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f559a700 1 -- 172.21.15.146:0/1439632535 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011630 con 0x7f55f8159e80 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f559a700 1 -- 172.21.15.146:0/1439632535 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec0117b0 con 0x7f55f8159e80 2024-08-15T16:59:51.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55f559a700 1 -- 172.21.15.146:0/1439632535 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec011b70 con 0x7f55f8159e80 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 -- start start 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdf980683b0 0x7fdf9806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf9806e450 0x7fdf98156b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9806e8b0 con 0x7fdf98155670 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf98156700 con 0x7fdf980683b0 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf97fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf98155af0 con 0x7fdf9806e450 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 Processor -- start 2024-08-15T16:59:51.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f05cf0a7700 1 -- start start 2024-08-15T16:59:51.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f743cf2f700 1 Processor -- start 2024-08-15T16:59:51.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f743cf2f700 1 -- start start 2024-08-15T16:59:51.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf96ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdf98155670 0x7fdf9813e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60110/0 (socket says 172.21.15.146:60110) 2024-08-15T16:59:51.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf96ffd700 1 -- 172.21.15.146:0/646292449 learned_addr learned my addr 172.21.15.146:0/646292449 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf967fc700 1 --2- 172.21.15.146:0/646292449 >> v2:172.21.15.161:3300/0 conn(0x7fdf980683b0 0x7fdf9806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf977fe700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf9806e450 0x7fdf98156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- 172.21.15.146:0/1439632535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 msgr2=0x7f55f80ade10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- 172.21.15.146:0/1439632535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80ade10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55f806fbc0 tx=0x7f55ec0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf95ffb700 1 -- 172.21.15.146:0/646292449 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857482499 0 0) 0x7fdf90001490 con 0x7fdf98155670 2024-08-15T16:59:51.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf95ffb700 1 -- 172.21.15.146:0/646292449 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003570 con 0x7fdf98155670 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- 172.21.15.146:0/1439632535 shutdown_connections 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- 172.21.15.146:0/1439632535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80ade10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- 172.21.15.146:0/1439632535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f80ac490 0x7f55f8159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 --2- 172.21.15.146:0/1439632535 >> v2:172.21.15.161:3300/0 conn(0x7f55f806e680 0x7f55f806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7f55fce72700 1 -- 172.21.15.146:0/1439632535 >> 172.21.15.146:0/1439632535 conn(0x7f55f8067810 msgr2=0x7f55f8067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf967fc700 1 -- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf9806e450 msgr2=0x7fdf98156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf967fc700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf9806e450 0x7fdf98156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf967fc700 1 -- 172.21.15.146:0/646292449 >> v1:172.21.15.146:6789/0 conn(0x7fdf98155670 legacy=0x7fdf9813e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.513+0000 7fdf967fc700 1 -- 172.21.15.146:0/646292449 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98155dd0 con 0x7fdf980683b0 2024-08-15T16:59:51.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- 172.21.15.146:0/1439632535 shutdown_connections 2024-08-15T16:59:51.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- 172.21.15.146:0/1439632535 wait complete. 2024-08-15T16:59:51.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c80b1950 0x7f05c8157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c80cacc0 0x7f05c80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8156bd0 con 0x7f05c813bf10 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8156eb0 con 0x7f05c80b1950 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c8157190 con 0x7f05c80cacc0 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 Processor -- start 2024-08-15T16:59:51.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf967fc700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf980683b0 0x7fdf9806df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800bb90 tx=0x7fdf8800b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26b266ca7a60930 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 -- start start 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49754/0 (socket says 172.21.15.146:49754) 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 -- 172.21.15.146:0/3306567294 learned_addr learned my addr 172.21.15.146:0/3306567294 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cd644700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c80cacc0 0x7f05c80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cce43700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c80b1950 0x7f05c8157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 -- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c80cacc0 msgr2=0x7f05c80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c80cacc0 0x7f05c80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 -- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c80b1950 msgr2=0x7f05c8157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c80b1950 0x7f05c8157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 -- 172.21.15.146:0/3306567294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009cb0 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c7fff700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00c980 tx=0x7f05bc001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc007360 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc0074e0 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004830 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- 172.21.15.146:0/3306567294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c8157470 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- 172.21.15.146:0/3306567294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c8158190 con 0x7f05c813bf10 2024-08-15T16:59:51.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc000153a70 0x7fc000159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f64659e6700 1 Processor -- start 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 Processor -- start 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc000153e40 0x7fc00006e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc000157810 con 0x7fc00015a180 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc000157dd0 con 0x7fc000153a70 2024-08-15T16:59:51.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fc00531f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00015a700 con 0x7fc000153e40 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f743cf2f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f743806e620 0x7f7438171c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f743cf2f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380af2d0 con 0x7f74380aeb80 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f743cf2f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74381666d0 con 0x7f74380b1550 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f743cf2f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74381669b0 con 0x7f743806e620 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f7436ffd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f74380b1550 0x7f743806df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42070/0 (socket says 172.21.15.146:42070) 2024-08-15T16:59:51.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf95ffb700 1 -- 172.21.15.146:0/646292449 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800db40 con 0x7fdf980683b0 2024-08-15T16:59:51.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f7436ffd700 1 -- 172.21.15.146:0/3631718720 learned_addr learned my addr 172.21.15.146:0/3631718720 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073849280 0 0) 0x7f74381666d0 con 0x7f74380b1550 2024-08-15T16:59:51.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7420003570 con 0x7f74380b1550 2024-08-15T16:59:51.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbfff7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc00015a180 0x7fc0000a9e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60128/0 (socket says 172.21.15.146:60128) 2024-08-15T16:59:51.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbfff7fe700 1 -- 172.21.15.146:0/1204462539 learned_addr learned my addr 172.21.15.146:0/1204462539 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbffeffd700 1 --2- 172.21.15.146:0/1204462539 >> v2:172.21.15.146:3300/0 conn(0x7fc000153a70 0x7fc000159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbffeffd700 1 -- 172.21.15.146:0/1204462539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc000153e40 msgr2=0x7fc00006e2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbffeffd700 1 --2- 172.21.15.146:0/1204462539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc000153e40 0x7fc00006e2b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677495358 0 0) 0x7f7420003570 con 0x7f74380b1550 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 >> v2:172.21.15.161:3300/0 conn(0x7f743806e620 msgr2=0x7f7438171c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f7437fff700 1 --2- 172.21.15.146:0/3631718720 >> v2:172.21.15.161:3300/0 conn(0x7f743806e620 0x7f7438171c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05cf0a7700 1 -- 172.21.15.146:0/3306567294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a80052b0 con 0x7f05c813bf10 2024-08-15T16:59:51.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- start start 2024-08-15T16:59:51.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf95ffb700 1 -- 172.21.15.146:0/646292449 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8800dca0 con 0x7fdf980683b0 2024-08-15T16:59:51.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf95ffb700 1 -- 172.21.15.146:0/646292449 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88019690 con 0x7fdf980683b0 2024-08-15T16:59:51.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 -- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf980683b0 msgr2=0x7fdf9806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf980683b0 0x7fdf9806df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800bb90 tx=0x7fdf8800b430 comp rx=0 tx=0).stop 2024-08-15T16:59:51.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 --2- 172.21.15.146:0/3631718720 >> v2:172.21.15.161:3300/0 conn(0x7f743806e620 0x7f7438171c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 >> v1:172.21.15.146:6789/0 conn(0x7f74380aeb80 legacy=0x7f7438165ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438166c90 con 0x7f74380b1550 2024-08-15T16:59:51.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f74300017f0 con 0x7f74380b1550 2024-08-15T16:59:51.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f806e680 0x7f55f80ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbffeffd700 1 -- 172.21.15.146:0/1204462539 >> v1:172.21.15.146:6789/0 conn(0x7fc00015a180 legacy=0x7fc0000a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fbffeffd700 1 -- 172.21.15.146:0/1204462539 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00015a9e0 con 0x7fc000153a70 2024-08-15T16:59:51.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c9df0 con 0x7f55f8159e80 2024-08-15T16:59:51.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80ca7c0 con 0x7f55f80ac490 2024-08-15T16:59:51.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55fce72700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c91e0 con 0x7f55f806e680 2024-08-15T16:59:51.547 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 -- 172.21.15.146:0/646292449 shutdown_connections 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 >> v1:172.21.15.161:6789/0 conn(0x7f74380b1550 legacy=0x7f743806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7420003a60 0x7f7420003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7420004810 0x7f7420004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 0x7f742000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381669b0 con 0x7f7420003a60 2024-08-15T16:59:51.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742000c230 con 0x7f7420004810 2024-08-15T16:59:51.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf9806e450 0x7fdf98156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 --2- 172.21.15.146:0/646292449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf980683b0 0x7fdf9806df10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 -- 172.21.15.146:0/646292449 >> 172.21.15.146:0/646292449 conn(0x7fdf980678d0 msgr2=0x7fdf98067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742000c3f0 con 0x7f74200053c0 2024-08-15T16:59:51.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f64659e6700 1 -- start start 2024-08-15T16:59:51.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 -- 172.21.15.146:0/646292449 shutdown_connections 2024-08-15T16:59:51.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55f5d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34194/0 (socket says 172.21.15.146:34194) 2024-08-15T16:59:51.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f55f5d9b700 1 -- 172.21.15.146:0/2543289549 learned_addr learned my addr 172.21.15.146:0/2543289549 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f7436ffd700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7420004810 0x7f7420004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7fdf97fff700 1 -- 172.21.15.146:0/646292449 wait complete. 2024-08-15T16:59:51.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f05bc004c40 con 0x7f05c813bf10 2024-08-15T16:59:51.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fbffeffd700 1 --2- 172.21.15.146:0/1204462539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153a70 0x7fc000159c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009e60 tx=0x7fbff8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275b604622ae51e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f05c4ff9700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 0x7f05b4062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05bc07bfb0 con 0x7f05c813bf10 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f05cce43700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 0x7f05b4062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f74377fe700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7420003a60 0x7f7420003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f7437fff700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 0x7f742000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 -- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7420004810 msgr2=0x7f7420004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7420004810 0x7f7420004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 -- 172.21.15.146:0/3631718720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7420003a60 msgr2=0x7f7420003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7420003a60 0x7f7420003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 -- 172.21.15.146:0/3631718720 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420003570 con 0x7f74200053c0 2024-08-15T16:59:51.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 -- 172.21.15.146:0/3631718720 reap_dead start 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f6d9d700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 -- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f806e680 msgr2=0x7f55f80ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f806e680 0x7f55f80ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 -- 172.21.15.146:0/2543289549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 msgr2=0x7f55f80a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 -- 172.21.15.146:0/2543289549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec009420 con 0x7f55f80ac490 2024-08-15T16:59:51.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f6d9d700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55f5d9b700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ed0 tx=0x7f55f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000ca70 con 0x7f55f80ac490 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55fce72700 1 -- 172.21.15.146:0/2543289549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f80c94c0 con 0x7f55f80ac490 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000f040 con 0x7f55f80ac490 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f000ce20 con 0x7f55f80ac490 2024-08-15T16:59:51.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55fce72700 1 -- 172.21.15.146:0/2543289549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f80c9a80 con 0x7f55f80ac490 2024-08-15T16:59:51.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f7437fff700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 0x7f742000ba60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f743806ea20 tx=0x7f742c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec7440ddd7299f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f64659e6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64600caeb0 0x7f64600ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f64659e6700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600ac060 con 0x7f64600b14a0 2024-08-15T16:59:51.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f64659e6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600cb5b0 con 0x7f646015a480 2024-08-15T16:59:51.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f64659e6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600caae0 con 0x7f64600caeb0 2024-08-15T16:59:51.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64600caeb0 0x7f64600ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64600caeb0 0x7f64600ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34212/0 (socket says 172.21.15.146:34212) 2024-08-15T16:59:51.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 -- 172.21.15.146:0/4157725121 learned_addr learned my addr 172.21.15.146:0/4157725121 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645e7fc700 1 --1- 172.21.15.146:0/4157725121 >> v1:172.21.15.146:6789/0 conn(0x7f64600b14a0 0x7f64600ac4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60144/0 (socket says 172.21.15.146:60144) 2024-08-15T16:59:51.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f74377fe700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7420003a60 0x7f7420003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:51.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0159d0 con 0x7f74200053c0 2024-08-15T16:59:51.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 Processor -- start 2024-08-15T16:59:51.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 -- start start 2024-08-15T16:59:51.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c015b70 con 0x7f74200053c0 2024-08-15T16:59:51.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f74367fc700 1 -- 172.21.15.146:0/3631718720 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00c600 con 0x7f74200053c0 2024-08-15T16:59:51.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f05cce43700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 0x7f05b4062d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05c8064090 tx=0x7f05b8018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 -- 172.21.15.146:0/4157725121 >> v1:172.21.15.161:6789/0 conn(0x7f646015a480 legacy=0x7f64600b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 -- 172.21.15.146:0/4157725121 >> v1:172.21.15.146:6789/0 conn(0x7f64600b14a0 legacy=0x7f64600ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 -- 172.21.15.146:0/4157725121 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600cb890 con 0x7f64600caeb0 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645f7fe700 1 --2- 172.21.15.146:0/4157725121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64600caeb0 0x7f64600ca5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6454013260 tx=0x7f6454011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec8a57c12aff350 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645dffb700 1 -- 172.21.15.146:0/4157725121 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454016070 con 0x7f64600caeb0 2024-08-15T16:59:51.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f9e27fff700 1 Processor -- start 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.517+0000 7f9e27fff700 1 -- start start 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e27fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9e2806e1a0 0x7f9e280aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e27fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e28166dc0 con 0x7f9e2806e570 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e27fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e281661e0 con 0x7f9e28165e10 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e27fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e28166490 con 0x7f9e2806e1a0 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e267fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9e2806e570 0x7f9e28165700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60158/0 (socket says 172.21.15.146:60158) 2024-08-15T16:59:51.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e267fc700 1 -- 172.21.15.146:0/3375301840 learned_addr learned my addr 172.21.15.146:0/3375301840 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e26ffd700 1 --2- 172.21.15.146:0/3375301840 >> v2:172.21.15.146:3300/0 conn(0x7f9e2806e1a0 0x7f9e280aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 0x7fdf9813ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf9806e450 0x7fdf9816d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf98151a40 con 0x7fdf9816da50 2024-08-15T16:59:51.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9812fb00 con 0x7fdf9806e450 2024-08-15T16:59:51.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf97fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf98131ba0 con 0x7fdf980683b0 2024-08-15T16:59:51.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d00052b0 con 0x7f55f80ac490 2024-08-15T16:59:51.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 -- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 msgr2=0x7f742000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 0x7f742000ba60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f743806ea20 tx=0x7f742c008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3375301840 >> v1:172.21.15.161:6789/0 conn(0x7f9e28165e10 legacy=0x7f9e28167250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3375301840 >> v1:172.21.15.146:6789/0 conn(0x7f9e2806e570 legacy=0x7f9e28165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3375301840 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28166770 con 0x7f9e2806e1a0 2024-08-15T16:59:51.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 -- 172.21.15.146:0/3631718720 shutdown_connections 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 --2- 172.21.15.146:0/3631718720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74200053c0 0x7f742000ba60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 -- 172.21.15.146:0/3631718720 >> 172.21.15.146:0/3631718720 conn(0x7f74380678d0 msgr2=0x7f7438067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf96ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 0x7fdf9813ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49790/0 (socket says 172.21.15.146:49790) 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf96ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 0x7fdf9813ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59682/0 (socket says 172.21.15.146:59682) 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 -- 172.21.15.146:0/1246372829 learned_addr learned my addr 172.21.15.146:0/1246372829 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f9e26ffd700 1 --2- 172.21.15.146:0/3375301840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e280aea80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009410 tx=0x7f9e180095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd182f9e895eae78 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 -- 172.21.15.146:0/3631718720 shutdown_connections 2024-08-15T16:59:51.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f743cf2f700 1 -- 172.21.15.146:0/3631718720 wait complete. 2024-08-15T16:59:51.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645dffb700 1 -- 172.21.15.146:0/4157725121 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64540110d0 con 0x7f64600caeb0 2024-08-15T16:59:51.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7f645dffb700 1 -- 172.21.15.146:0/4157725121 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011450 con 0x7f64600caeb0 2024-08-15T16:59:51.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f55f0036bb0 con 0x7f55f80ac490 2024-08-15T16:59:51.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f55e67fc700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 0x7f55d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55f007d6e0 con 0x7f55f80ac490 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 -- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 msgr2=0x7fdf9813ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 0x7fdf9813ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 -- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf9806e450 msgr2=0x7fdf9816d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf9806e450 0x7fdf9816d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.521+0000 7fdf977fe700 1 -- 172.21.15.146:0/1246372829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8800b6d0 con 0x7fdf9816da50 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fdf977fe700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d8c0 tx=0x7fdf8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f55f659c700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 0x7f55d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- 172.21.15.146:0/4157725121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64600caeb0 msgr2=0x7f64600ca5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 --2- 172.21.15.146:0/4157725121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64600caeb0 0x7f64600ca5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6454013260 tx=0x7f6454011af0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- 172.21.15.146:0/4157725121 shutdown_connections 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 --2- 172.21.15.146:0/4157725121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64600caeb0 0x7f64600ca5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- 172.21.15.146:0/4157725121 >> 172.21.15.146:0/4157725121 conn(0x7f64600678d0 msgr2=0x7f6460067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- 172.21.15.146:0/4157725121 shutdown_connections 2024-08-15T16:59:51.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- 172.21.15.146:0/4157725121 wait complete. 2024-08-15T16:59:51.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 Processor -- start 2024-08-15T16:59:51.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f55f659c700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 0x7f55d8062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009bb0 tx=0x7f55e80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 -- start start 2024-08-15T16:59:51.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 Processor -- start 2024-08-15T16:59:51.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 -- start start 2024-08-15T16:59:51.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f18380b0350 0x7f18380ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380af2b0 con 0x7f1838155d20 2024-08-15T16:59:51.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18380af590 con 0x7f183806df10 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f183ce15700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18380b07a0 con 0x7f18380b0350 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74200053c0 0x7f743815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7438063060 0x7f74380634d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438063a10 con 0x7f74200053c0 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438179860 con 0x7f7438176540 2024-08-15T16:59:51.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f743cf2f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743817a3e0 con 0x7f7438063060 2024-08-15T16:59:51.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 Processor -- start 2024-08-15T16:59:51.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f18377fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1838155d20 0x7f18380aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60166/0 (socket says 172.21.15.146:60166) 2024-08-15T16:59:51.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f18377fe700 1 -- 172.21.15.146:0/1517928911 learned_addr learned my addr 172.21.15.146:0/1517928911 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:51.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f1836ffd700 1 --2- 172.21.15.146:0/1517928911 >> v2:172.21.15.146:3300/0 conn(0x7f18380b0350 0x7f18380ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 -- start start 2024-08-15T16:59:51.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7436ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7438063060 0x7f74380634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34224/0 (socket says 172.21.15.146:34224) 2024-08-15T16:59:51.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 -- 172.21.15.146:0/2605442772 learned_addr learned my addr 172.21.15.146:0/2605442772 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f05bc04c3e0 con 0x7f05c813bf10 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f1836ffd700 1 -- 172.21.15.146:0/1517928911 >> v1:172.21.15.161:6789/0 conn(0x7f183806df10 legacy=0x7f183806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f1836ffd700 1 -- 172.21.15.146:0/1517928911 >> v1:172.21.15.146:6789/0 conn(0x7f1838155d20 legacy=0x7f18380aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f1836ffd700 1 -- 172.21.15.146:0/1517928911 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18380af870 con 0x7f18380b0350 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f74377fe700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74200053c0 0x7f743815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 -- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7438063060 msgr2=0x7f74380634d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7438063060 0x7f74380634d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 -- 172.21.15.146:0/2605442772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74200053c0 msgr2=0x7f743815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74200053c0 0x7f743815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 -- 172.21.15.146:0/2605442772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438179ab0 con 0x7f7438176540 2024-08-15T16:59:51.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0125f0 con 0x7fdf9816da50 2024-08-15T16:59:51.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f9e25ffb700 1 -- 172.21.15.146:0/3375301840 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800f750 con 0x7f9e2806e1a0 2024-08-15T16:59:51.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f1836ffd700 1 --2- 172.21.15.146:0/1517928911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18380b0350 0x7f18380ad7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098e0 tx=0x7f182c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e66f2d5d822fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fdf97fff700 1 -- 172.21.15.146:0/1246372829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf98131270 con 0x7fdf9816da50 2024-08-15T16:59:51.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fdf97fff700 1 -- 172.21.15.146:0/1246372829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf98131830 con 0x7fdf9816da50 2024-08-15T16:59:51.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c012750 con 0x7fdf9816da50 2024-08-15T16:59:51.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f18367fc700 1 -- 172.21.15.146:0/1517928911 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c009e80 con 0x7f18380b0350 2024-08-15T16:59:51.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f18367fc700 1 -- 172.21.15.146:0/1517928911 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00b860 con 0x7f18380b0350 2024-08-15T16:59:51.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f18367fc700 1 -- 172.21.15.146:0/1517928911 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00bbe0 con 0x7f18380b0350 2024-08-15T16:59:51.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f64659e6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64600caeb0 0x7f646014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f646015a480 0x7f646014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646014c850 con 0x7f64600caeb0 2024-08-15T16:59:51.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646014ba20 con 0x7f646015a480 2024-08-15T16:59:51.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64601527f0 con 0x7f64600b14a0 2024-08-15T16:59:51.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c012b10 con 0x7fdf9816da50 2024-08-15T16:59:51.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7f7437fff700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f743806f250 tx=0x7f742c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e25ffb700 1 -- 172.21.15.146:0/3375301840 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1800f8b0 con 0x7f9e2806e1a0 2024-08-15T16:59:51.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e25ffb700 1 -- 172.21.15.146:0/3375301840 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800fc30 con 0x7f9e2806e1a0 2024-08-15T16:59:51.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f183ce15700 1 -- 172.21.15.146:0/1517928911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18380b0350 msgr2=0x7f18380ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f183ce15700 1 --2- 172.21.15.146:0/1517928911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18380b0350 0x7f18380ad7c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f182c0098e0 tx=0x7f182c0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f183ce15700 1 -- 172.21.15.146:0/1517928911 shutdown_connections 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f183ce15700 1 --2- 172.21.15.146:0/1517928911 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f18380b0350 0x7f18380ad7c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f183ce15700 1 -- 172.21.15.146:0/1517928911 >> 172.21.15.146:0/1517928911 conn(0x7f18380678d0 msgr2=0x7f1838067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf97fff700 1 -- 172.21.15.146:0/1246372829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf780052b0 con 0x7fdf9816da50 2024-08-15T16:59:51.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdf8c01e460 con 0x7fdf9816da50 2024-08-15T16:59:51.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0159d0 con 0x7f7438176540 2024-08-15T16:59:51.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00ce70 con 0x7f7438176540 2024-08-15T16:59:51.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f55f004da90 con 0x7f55f80ac490 2024-08-15T16:59:51.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59698/0 (socket says 172.21.15.146:59698) 2024-08-15T16:59:51.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 -- 172.21.15.146:0/2829328351 learned_addr learned my addr 172.21.15.146:0/2829328351 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 -- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f646015a480 msgr2=0x7f646014dda0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f646015a480 0x7f646014dda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 -- 172.21.15.146:0/2829328351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64600caeb0 msgr2=0x7f646014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64600caeb0 0x7f646014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 -- 172.21.15.146:0/2829328351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454011d90 con 0x7f64600b14a0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f645effd700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f645000c490 tx=0x7f645000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011590 con 0x7f64600b14a0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450011710 con 0x7f64600b14a0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6460152040 con 0x7f64600b14a0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011ad0 con 0x7f64600b14a0 2024-08-15T16:59:51.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460151d60 con 0x7f64600b14a0 2024-08-15T16:59:51.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 -- 172.21.15.146:0/3375301840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 msgr2=0x7f9e280aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3375301840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e280aea80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009410 tx=0x7f9e180095f0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf7effd700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 0x7fdf84062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf8c07d100 con 0x7fdf9816da50 2024-08-15T16:59:51.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf96ffd700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 0x7fdf84062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 -- 172.21.15.146:0/3375301840 shutdown_connections 2024-08-15T16:59:51.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3375301840 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e280aea80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 -- 172.21.15.146:0/3375301840 >> 172.21.15.146:0/3375301840 conn(0x7f9e280678d0 msgr2=0x7f9e28067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c01eb00 con 0x7f7438176540 2024-08-15T16:59:51.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7438179d90 con 0x7f7438176540 2024-08-15T16:59:51.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7fdf96ffd700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 0x7fdf84062d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf900015f0 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64400052b0 con 0x7f64600b14a0 2024-08-15T16:59:51.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743815dad0 con 0x7f7438176540 2024-08-15T16:59:51.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 -- 172.21.15.146:0/3375301840 shutdown_connections 2024-08-15T16:59:51.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.529+0000 7f9e27fff700 1 -- 172.21.15.146:0/3375301840 wait complete. 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fbffdffb700 1 -- 172.21.15.146:0/1204462539 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800bc30 con 0x7fc000153a70 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fbffdffb700 1 -- 172.21.15.146:0/1204462539 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff800bdb0 con 0x7fc000153a70 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.525+0000 7fbffdffb700 1 -- 172.21.15.146:0/1204462539 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80116b0 con 0x7fc000153a70 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7fc00531f700 1 -- 172.21.15.146:0/1204462539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153a70 msgr2=0x7fc000159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7fc00531f700 1 --2- 172.21.15.146:0/1204462539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153a70 0x7fc000159c40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009e60 tx=0x7fbff8009780 comp rx=0 tx=0).stop 2024-08-15T16:59:51.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f742c01c3e0 con 0x7f7438176540 2024-08-15T16:59:51.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- 172.21.15.146:0/1517928911 shutdown_connections 2024-08-15T16:59:51.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- 172.21.15.146:0/1517928911 wait complete. 2024-08-15T16:59:51.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f741f7fe700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 0x7f7424062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f742c085f30 con 0x7f7438176540 2024-08-15T16:59:51.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f74377fe700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 0x7f7424062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f74377fe700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 0x7f7424062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74280010e0 tx=0x7f7428009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 Processor -- start 2024-08-15T16:59:51.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 Processor -- start 2024-08-15T16:59:51.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- start start 2024-08-15T16:59:51.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 -- start start 2024-08-15T16:59:51.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e2806e570 0x7f9e28162410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e28165e10 0x7f9e28162950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28148ec0 con 0x7f9e2806e1a0 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e28176e80 con 0x7f9e28165e10 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e27fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e280a6a90 con 0x7f9e2806e570 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49828/0 (socket says 172.21.15.146:49828) 2024-08-15T16:59:51.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f183806df10 0x7f183814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3150087317 learned_addr learned my addr 172.21.15.146:0/3150087317 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e277fe700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e28165e10 0x7f9e28162950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e267fc700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e2806e570 0x7f9e28162410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e2806e570 msgr2=0x7f9e28162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e2806e570 0x7f9e28162410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18380b0350 0x7f183814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e28165e10 msgr2=0x7f9e28162950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e28165e10 0x7f9e28162950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 -- 172.21.15.146:0/3150087317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18009160 con 0x7f9e2806e1a0 2024-08-15T16:59:51.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f9e26ffd700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009e10 tx=0x7f9e18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18381564f0 con 0x7f1838155d20 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1838156ab0 con 0x7f183806df10 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f183ce15700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f183814e800 con 0x7f18380b0350 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f1837fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f18377fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f183806df10 0x7f183814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f1837fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49840/0 (socket says 172.21.15.146:49840) 2024-08-15T16:59:51.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.533+0000 7f1837fff700 1 -- 172.21.15.146:0/2334925272 learned_addr learned my addr 172.21.15.146:0/2334925272 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1836ffd700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18380b0350 0x7f183814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f645001b3b0 con 0x7f64600b14a0 2024-08-15T16:59:51.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdf8c04d550 con 0x7fdf9816da50 2024-08-15T16:59:51.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 -- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18380b0350 msgr2=0x7f183814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18380b0350 0x7f183814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 -- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f183806df10 msgr2=0x7f183814f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f183806df10 0x7f183814f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 -- 172.21.15.146:0/2334925272 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c009420 con 0x7f1838155d20 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800fc30 con 0x7f9e2806e1a0 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1800f2a0 con 0x7f9e2806e1a0 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18023690 con 0x7f9e2806e1a0 2024-08-15T16:59:51.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e280a6d70 con 0x7f9e2806e1a0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e280a7330 con 0x7f9e2806e1a0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9e1800b450 con 0x7f9e2806e1a0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 0x7f9e14062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e18013070 con 0x7f9e2806e1a0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.541+0000 7f9e267fc700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 0x7f9e14062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.541+0000 7f9e267fc700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 0x7f9e14062de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e28147ca0 tx=0x7f9e20001a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f1837fff700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f182800c4c0 tx=0x7f1828007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182800ce70 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1828011590 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f183814bea0 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828011970 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1838156740 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f1828011d20 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18180052b0 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 0x7f1824062e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f182807be70 con 0x7f1838155d20 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.541+0000 7f18377fe700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 0x7f1824062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.541+0000 7f18377fe700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 0x7f1824062e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18300020c0 tx=0x7f1830001a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74380643a0 con 0x7f7438176540 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f6446ffd700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 0x7f644c062d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f645007c150 con 0x7f64600b14a0 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f645e7fc700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 0x7f644c062d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.537+0000 7f645e7fc700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 0x7f644c062d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6460067f60 tx=0x7f6458001750 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.541+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f645004c500 con 0x7f64600b14a0 2024-08-15T16:59:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f742c02b020 con 0x7f7438176540 2024-08-15T16:59:51.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- 172.21.15.146:0/1204462539 shutdown_connections 2024-08-15T16:59:51.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 --2- 172.21.15.146:0/1204462539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc000153e40 0x7fc00006e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 --2- 172.21.15.146:0/1204462539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153a70 0x7fc000159c40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- 172.21.15.146:0/1204462539 >> 172.21.15.146:0/1204462539 conn(0x7fc000067810 msgr2=0x7fc000067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- 172.21.15.146:0/1204462539 shutdown_connections 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- 172.21.15.146:0/1204462539 wait complete. 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 Processor -- start 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- start start 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 0x7fc000138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00015a180 0x7fc0000991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000afbf0 con 0x7fc000153e40 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc000099700 con 0x7fc000153a70 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fc00531f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000af300 con 0x7fc00015a180 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fbffe7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fbffeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 0x7fc000138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fbffe7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49842/0 (socket says 172.21.15.146:49842) 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fbffeffd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 0x7fc000138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34254/0 (socket says 172.21.15.146:34254) 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.545+0000 7fbffe7fc700 1 -- 172.21.15.146:0/1114249687 learned_addr learned my addr 172.21.15.146:0/1114249687 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 -- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00015a180 msgr2=0x7fc0000991c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00015a180 0x7fc0000991c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 -- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 msgr2=0x7fc000138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 0x7fc000138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 -- 172.21.15.146:0/1114249687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009a20 con 0x7fc000153e40 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7fbffe7fc700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b7e0 tx=0x7fbff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.549+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e28176c80 con 0x7f9e2806e1a0 2024-08-15T16:59:51.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.553+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f182804c2a0 con 0x7f1838155d20 2024-08-15T16:59:51.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400f510 con 0x7fc000153e40 2024-08-15T16:59:51.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9e1804e760 con 0x7f9e2806e1a0 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000af5a0 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc000138ff0 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff400f670 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400fa30 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.557+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbff401a400 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.561+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe00052b0 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.561+0000 7fbfe6ffd700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 0x7fbfec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.561+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbff407c570 con 0x7fc000153e40 2024-08-15T16:59:51.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.561+0000 7fbffeffd700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 0x7fbfec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.561+0000 7fbffeffd700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 0x7fbfec062d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009e60 tx=0x7fbff801b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.569+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fbff404c9c0 con 0x7fc000153e40 2024-08-15T16:59:51.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.817+0000 7f05cf0a7700 1 -- 172.21.15.146:0/3306567294 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f05a8005560 con 0x7f05c813bf10 2024-08-15T16:59:51.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.817+0000 7f05c4ff9700 1 -- 172.21.15.146:0/3306567294 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05bc018020 con 0x7f05c813bf10 2024-08-15T16:59:51.831 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 msgr2=0x7f05b4062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 0x7f05b4062d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f05c8064090 tx=0x7f05b8018000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 msgr2=0x7f05c80ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00c980 tx=0x7f05bc001b70 comp rx=0 tx=0).stop 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 shutdown_connections 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f05b40608e0 0x7f05b4062d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05c80cacc0 0x7f05c80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05c813bf10 0x7f05c80ca780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 --2- 172.21.15.146:0/3306567294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f05c80b1950 0x7f05c8157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 >> 172.21.15.146:0/3306567294 conn(0x7f05c80678d0 msgr2=0x7f05c8154aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 shutdown_connections 2024-08-15T16:59:51.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.825+0000 7f05ae7fc700 1 -- 172.21.15.146:0/3306567294 wait complete. 2024-08-15T16:59:51.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.833+0000 7fdf97fff700 1 -- 172.21.15.146:0/1246372829 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdf78005830 con 0x7fdf9816da50 2024-08-15T16:59:51.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.837+0000 7fdf7effd700 1 -- 172.21.15.146:0/1246372829 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf8c050b70 con 0x7fdf9816da50 2024-08-15T16:59:51.850 INFO:teuthology.orchestra.run.smithi146.stdout:60129542149 2024-08-15T16:59:51.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-08-15T16:59:51.852 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.852 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.837+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 msgr2=0x7fdf84062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.837+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 0x7fdf84062d40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdf900015f0 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.837+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 msgr2=0x7fdf9812f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.837+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00d8c0 tx=0x7fdf8c008040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 shutdown_connections 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf84060890 0x7fdf84062d40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf9816da50 0x7fdf9812f000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf9806e450 0x7fdf9816d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 --2- 172.21.15.146:0/1246372829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf980683b0 0x7fdf9813ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 >> 172.21.15.146:0/1246372829 conn(0x7fdf980678d0 msgr2=0x7fdf98135830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 shutdown_connections 2024-08-15T16:59:51.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.841+0000 7fdf7cff9700 1 -- 172.21.15.146:0/1246372829 wait complete. 2024-08-15T16:59:51.855 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.845+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f55d0005830 con 0x7f55f80ac490 2024-08-15T16:59:51.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.845+0000 7f55e67fc700 1 -- 172.21.15.146:0/2543289549 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55f00510b0 con 0x7f55f80ac490 2024-08-15T16:59:51.856 INFO:teuthology.orchestra.run.smithi146.stdout:60129542151 2024-08-15T16:59:51.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 msgr2=0x7f55d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.859 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 0x7f55d8062d10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009bb0 tx=0x7f55e80091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 msgr2=0x7f55f80c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ed0 tx=0x7f55f000b040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 shutdown_connections 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f55d8060860 0x7f55d8062d10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55f8159e80 0x7f55f80a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f80ac490 0x7f55f80c8870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 --2- 172.21.15.146:0/2543289549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f806e680 0x7f55f80ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 >> 172.21.15.146:0/2543289549 conn(0x7f55f8067810 msgr2=0x7f55f806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 shutdown_connections 2024-08-15T16:59:51.861 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.849+0000 7f55d7fff700 1 -- 172.21.15.146:0/2543289549 wait complete. 2024-08-15T16:59:51.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1818005560 con 0x7f1838155d20 2024-08-15T16:59:51.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f743817a520 con 0x7f7438176540 2024-08-15T16:59:51.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6440005560 con 0x7f64600b14a0 2024-08-15T16:59:51.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f181f7fe700 1 -- 172.21.15.146:0/2334925272 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f182804f8c0 con 0x7f1838155d20 2024-08-15T16:59:51.866 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f741f7fe700 1 -- 172.21.15.146:0/2605442772 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f742c0563a0 con 0x7f7438176540 2024-08-15T16:59:51.867 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-08-15T16:59:51.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.853+0000 7f6446ffd700 1 -- 172.21.15.146:0/2829328351 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f645004fb20 con 0x7f64600b14a0 2024-08-15T16:59:51.868 INFO:teuthology.orchestra.run.smithi146.stdout:60129542151 2024-08-15T16:59:51.868 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 msgr2=0x7f1824062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 0x7f1824062e60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18300020c0 tx=0x7f1830001a10 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 msgr2=0x7f1838166570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f182800c4c0 tx=0x7f1828007f50 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 shutdown_connections 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18240609b0 0x7f1824062e60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1838155d20 0x7f1838166570 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f18380b0350 0x7f183814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 --2- 172.21.15.146:0/2334925272 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f183806df10 0x7f183814f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 >> 172.21.15.146:0/2334925272 conn(0x7f18380678d0 msgr2=0x7f1838063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 shutdown_connections 2024-08-15T16:59:51.870 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f183ce15700 1 -- 172.21.15.146:0/2334925272 wait complete. 2024-08-15T16:59:51.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 msgr2=0x7f7424062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 0x7f7424062d10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74280010e0 tx=0x7f7428009040 comp rx=0 tx=0).stop 2024-08-15T16:59:51.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 msgr2=0x7f74381769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f743806f250 tx=0x7f742c01f000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 shutdown_connections 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7424060860 0x7f7424062d10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7438176540 0x7f74381769b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7438063060 0x7f74380634d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 --2- 172.21.15.146:0/2605442772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74200053c0 0x7f743815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 >> 172.21.15.146:0/2605442772 conn(0x7f74380678d0 msgr2=0x7f743816a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 shutdown_connections 2024-08-15T16:59:51.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f743cf2f700 1 -- 172.21.15.146:0/2605442772 wait complete. 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 msgr2=0x7f644c062d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 0x7f644c062d90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6460067f60 tx=0x7f6458001750 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 msgr2=0x7f64601522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f645000c490 tx=0x7f645000c850 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 shutdown_connections 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f644c0608e0 0x7f644c062d90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f646015a480 0x7f646014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f64600caeb0 0x7f646014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 --2- 172.21.15.146:0/2829328351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64600b14a0 0x7f64601522b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 >> 172.21.15.146:0/2829328351 conn(0x7f64600678d0 msgr2=0x7f6460155ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 shutdown_connections 2024-08-15T16:59:51.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.857+0000 7f64659e6700 1 -- 172.21.15.146:0/2829328351 wait complete. 2024-08-15T16:59:51.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.2 2024-08-15T16:59:51.877 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-08-15T16:59:51.881 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.869+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbfe0005830 con 0x7fc000153e40 2024-08-15T16:59:51.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.869+0000 7fbfe6ffd700 1 -- 172.21.15.146:0/1114249687 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbff404ffe0 con 0x7fc000153e40 2024-08-15T16:59:51.882 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.869+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9e280643a0 con 0x7f9e2806e1a0 2024-08-15T16:59:51.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.869+0000 7f9e0effd700 1 -- 172.21.15.146:0/3150087317 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e18051d80 con 0x7f9e2806e1a0 2024-08-15T16:59:51.883 INFO:teuthology.orchestra.run.smithi146.stdout:60129542150 2024-08-15T16:59:51.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-08-15T16:59:51.883 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 msgr2=0x7fbfec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 0x7fbfec062d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009e60 tx=0x7fbff801b000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 msgr2=0x7fc000098c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b7e0 tx=0x7fbff40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 shutdown_connections 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbfec060890 0x7fbfec062d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00015a180 0x7fc0000991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000153e40 0x7fc000098c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 --2- 172.21.15.146:0/1114249687 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc000153a70 0x7fc000138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 >> 172.21.15.146:0/1114249687 conn(0x7fc000067810 msgr2=0x7fc000154260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 shutdown_connections 2024-08-15T16:59:51.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7fc00531f700 1 -- 172.21.15.146:0/1114249687 wait complete. 2024-08-15T16:59:51.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-08-15T16:59:51.887 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 msgr2=0x7f9e14062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 0x7f9e14062de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e28147ca0 tx=0x7f9e20001a10 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 msgr2=0x7f9e2815e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009e10 tx=0x7f9e18015000 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 shutdown_connections 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e14060930 0x7f9e14062de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e28165e10 0x7f9e28162950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e2806e570 0x7f9e28162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 --2- 172.21.15.146:0/3150087317 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e2806e1a0 0x7f9e2815e5f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 >> 172.21.15.146:0/3150087317 conn(0x7f9e280678d0 msgr2=0x7f9e281656b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:51.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 shutdown_connections 2024-08-15T16:59:51.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.873+0000 7f9e27fff700 1 -- 172.21.15.146:0/3150087317 wait complete. 2024-08-15T16:59:51.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.6 2024-08-15T16:59:51.895 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.7 2024-08-15T16:59:51.896 DEBUG:teuthology.parallel:result is None 2024-08-15T16:59:51.896 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T16:59:51.896 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:51.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 Processor -- start 2024-08-15T16:59:51.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 -- start start 2024-08-15T16:59:51.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f69f4155d20 0x7f69f40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f40ad7c0 0x7f69f40aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69f406df10 0x7f69f406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f40adc10 con 0x7f69f406df10 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f406e8d0 con 0x7f69f4155d20 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fae99700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40af2b0 con 0x7f69f40ad7c0 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fa698700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69f406df10 0x7f69f406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.981+0000 7f69fa698700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f69f406df10 0x7f69f406e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49848/0 (socket says 172.21.15.146:49848) 2024-08-15T16:59:51.996 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 -- 172.21.15.146:0/831296121 learned_addr learned my addr 172.21.15.146:0/831296121 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69f9e97700 1 --2- 172.21.15.146:0/831296121 >> v2:172.21.15.161:3300/0 conn(0x7f69f4155d20 0x7f69f40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69f9696700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f40ad7c0 0x7f69f40aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 -- 172.21.15.146:0/831296121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f40ad7c0 msgr2=0x7f69f40aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f40ad7c0 0x7f69f40aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 -- 172.21.15.146:0/831296121 >> v2:172.21.15.161:3300/0 conn(0x7f69f4155d20 msgr2=0x7f69f40b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 --2- 172.21.15.146:0/831296121 >> v2:172.21.15.161:3300/0 conn(0x7f69f4155d20 0x7f69f40b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:51.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 -- 172.21.15.146:0/831296121 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f40af590 con 0x7f69f406df10 2024-08-15T16:59:51.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fa698700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f406e390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69f40b1d50 tx=0x7f69f000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dd1369a6d9525f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:51.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69f8e95700 1 -- 172.21.15.146:0/831296121 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000db00 con 0x7f69f406df10 2024-08-15T16:59:51.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69f8e95700 1 -- 172.21.15.146:0/831296121 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f000dc60 con 0x7f69f406df10 2024-08-15T16:59:51.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69f8e95700 1 -- 172.21.15.146:0/831296121 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0019480 con 0x7f69f406df10 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- 172.21.15.146:0/831296121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 msgr2=0x7f69f406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f406e390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f69f40b1d50 tx=0x7f69f000b4f0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- 172.21.15.146:0/831296121 shutdown_connections 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f406e390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 --2- 172.21.15.146:0/831296121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f40ad7c0 0x7f69f40aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 --2- 172.21.15.146:0/831296121 >> v2:172.21.15.161:3300/0 conn(0x7f69f4155d20 0x7f69f40b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- 172.21.15.146:0/831296121 >> 172.21.15.146:0/831296121 conn(0x7f69f40678d0 msgr2=0x7f69f4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- 172.21.15.146:0/831296121 shutdown_connections 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- 172.21.15.146:0/831296121 wait complete. 2024-08-15T16:59:52.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 Processor -- start 2024-08-15T16:59:52.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.985+0000 7f69fae99700 1 -- start start 2024-08-15T16:59:52.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f40ad7c0 0x7f69f406a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f4155d20 0x7f69f41547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f414ade0 con 0x7f69f406df10 2024-08-15T16:59:52.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41568e0 con 0x7f69f40ad7c0 2024-08-15T16:59:52.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41553b0 con 0x7f69f4155d20 2024-08-15T16:59:52.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49858/0 (socket says 172.21.15.146:49858) 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 -- 172.21.15.146:0/29219725 learned_addr learned my addr 172.21.15.146:0/29219725 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9696700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f40ad7c0 0x7f69f406a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fa698700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f4155d20 0x7f69f41547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 -- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f4155d20 msgr2=0x7f69f41547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f4155d20 0x7f69f41547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 -- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f40ad7c0 msgr2=0x7f69f406a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f40ad7c0 0x7f69f406a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 -- 172.21.15.146:0/29219725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f000b790 con 0x7f69f406df10 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69f9e97700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009ef0 tx=0x7f69ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec00cb70 con 0x7f69f406df10 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec00f040 con 0x7f69f406df10 2024-08-15T16:59:52.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f4156290 con 0x7f69f406df10 2024-08-15T16:59:52.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec014480 con 0x7f69f406df10 2024-08-15T16:59:52.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f41572b0 con 0x7f69f406df10 2024-08-15T16:59:52.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.989+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f41560f0 con 0x7f69f406df10 2024-08-15T16:59:52.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.993+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f69ec0147f0 con 0x7f69f406df10 2024-08-15T16:59:52.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.993+0000 7f69e9ffb700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 0x7f69d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.993+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f69ec07db30 con 0x7f69f406df10 2024-08-15T16:59:52.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.993+0000 7f69f9696700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 0x7f69d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.993+0000 7f69f9696700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 0x7f69d8062d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69f4069da0 tx=0x7f69e40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:51.997+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f69ec04df80 con 0x7f69f406df10 2024-08-15T16:59:52.157 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.145+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f41560f0 con 0x7f69d8060860 2024-08-15T16:59:52.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.161+0000 7f69e9ffb700 1 -- 172.21.15.146:0/29219725 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264493 (secure 0 0 0) 0x7f69f41560f0 con 0x7f69d8060860 2024-08-15T16:59:52.181 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:52.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.165+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 msgr2=0x7f69d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.184 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.165+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 0x7f69d8062d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69f4069da0 tx=0x7f69e40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.165+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 msgr2=0x7f69f4156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.165+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f69ec009ef0 tx=0x7f69ec00b040 comp rx=0 tx=0).stop 2024-08-15T16:59:52.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 shutdown_connections 2024-08-15T16:59:52.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f69d8060860 0x7f69d8062d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f69f4155d20 0x7f69f41547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f69f40ad7c0 0x7f69f406a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 --2- 172.21.15.146:0/29219725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f69f406df10 0x7f69f4156d70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 >> 172.21.15.146:0/29219725 conn(0x7f69f40678d0 msgr2=0x7f69f4068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 shutdown_connections 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.169+0000 7f69fae99700 1 -- 172.21.15.146:0/29219725 wait complete. 2024-08-15T16:59:52.186 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:52.193 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-15T16:59:51.646407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385912,"kb_used_data":2884,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588808,"statfs":{"total":773094113280,"available":770650939392,"internally_reserved":0,"allocated":2953216,"data_stored":1774763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.264320"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413334+0000","last_change":"2024-08-15T16:59:42.409245+0000","last_active":"2024-08-15T16:59:42.413334+0000","last_peered":"2024-08-15T16:59:42.413334+0000","last_clean":"2024-08-15T16:59:42.413334+0000","last_became_active":"2024-08-15T16:59:42.408987+0000","last_became_peered":"2024-08-15T16:59:42.408987+0000","last_unstale":"2024-08-15T16:59:42.413334+0000","last_undegraded":"2024-08-15T16:59:42.413334+0000","last_fullsized":"2024-08-15T16:59:42.413334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:01:07.089558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422299+0000","last_change":"2024-08-15T16:59:42.418957+0000","last_active":"2024-08-15T16:59:42.422299+0000","last_peered":"2024-08-15T16:59:42.422299+0000","last_clean":"2024-08-15T16:59:42.422299+0000","last_became_active":"2024-08-15T16:59:42.418575+0000","last_became_peered":"2024-08-15T16:59:42.418575+0000","last_unstale":"2024-08-15T16:59:42.422299+0000","last_undegraded":"2024-08-15T16:59:42.422299+0000","last_fullsized":"2024-08-15T16:59:42.422299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419909+0000","last_change":"2024-08-15T16:59:42.410104+0000","last_active":"2024-08-15T16:59:42.419909+0000","last_peered":"2024-08-15T16:59:42.419909+0000","last_clean":"2024-08-15T16:59:42.419909+0000","last_became_active":"2024-08-15T16:59:42.409813+0000","last_became_peered":"2024-08-15T16:59:42.409813+0000","last_unstale":"2024-08-15T16:59:42.419909+0000","last_undegraded":"2024-08-15T16:59:42.419909+0000","last_fullsized":"2024-08-15T16:59:42.419909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:00.063666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416866+0000","last_change":"2024-08-15T16:59:42.415914+0000","last_active":"2024-08-15T16:59:42.416866+0000","last_peered":"2024-08-15T16:59:42.416866+0000","last_clean":"2024-08-15T16:59:42.416866+0000","last_became_active":"2024-08-15T16:59:42.415646+0000","last_became_peered":"2024-08-15T16:59:42.415646+0000","last_unstale":"2024-08-15T16:59:42.416866+0000","last_undegraded":"2024-08-15T16:59:42.416866+0000","last_fullsized":"2024-08-15T16:59:42.416866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:10:45.984041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413099+0000","last_change":"2024-08-15T16:59:42.406837+0000","last_active":"2024-08-15T16:59:42.413099+0000","last_peered":"2024-08-15T16:59:42.413099+0000","last_clean":"2024-08-15T16:59:42.413099+0000","last_became_active":"2024-08-15T16:59:42.406562+0000","last_became_peered":"2024-08-15T16:59:42.406562+0000","last_unstale":"2024-08-15T16:59:42.413099+0000","last_undegraded":"2024-08-15T16:59:42.413099+0000","last_fullsized":"2024-08-15T16:59:42.413099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:07:47.921825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414600+0000","last_change":"2024-08-15T16:59:42.403625+0000","last_active":"2024-08-15T16:59:42.414600+0000","last_peered":"2024-08-15T16:59:42.414600+0000","last_clean":"2024-08-15T16:59:42.414600+0000","last_became_active":"2024-08-15T16:59:42.403339+0000","last_became_peered":"2024-08-15T16:59:42.403339+0000","last_unstale":"2024-08-15T16:59:42.414600+0000","last_undegraded":"2024-08-15T16:59:42.414600+0000","last_fullsized":"2024-08-15T16:59:42.414600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:30.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417227+0000","last_change":"2024-08-15T16:59:42.413170+0000","last_active":"2024-08-15T16:59:42.417227+0000","last_peered":"2024-08-15T16:59:42.417227+0000","last_clean":"2024-08-15T16:59:42.417227+0000","last_became_active":"2024-08-15T16:59:42.409523+0000","last_became_peered":"2024-08-15T16:59:42.409523+0000","last_unstale":"2024-08-15T16:59:42.417227+0000","last_undegraded":"2024-08-15T16:59:42.417227+0000","last_fullsized":"2024-08-15T16:59:42.417227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:25.911685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418927+0000","last_change":"2024-08-15T16:59:42.412418+0000","last_active":"2024-08-15T16:59:42.418927+0000","last_peered":"2024-08-15T16:59:42.418927+0000","last_clean":"2024-08-15T16:59:42.418927+0000","last_became_active":"2024-08-15T16:59:42.412174+0000","last_became_peered":"2024-08-15T16:59:42.412174+0000","last_unstale":"2024-08-15T16:59:42.418927+0000","last_undegraded":"2024-08-15T16:59:42.418927+0000","last_fullsized":"2024-08-15T16:59:42.418927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:00:19.102852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419430+0000","last_change":"2024-08-15T16:59:42.416969+0000","last_active":"2024-08-15T16:59:42.419430+0000","last_peered":"2024-08-15T16:59:42.419430+0000","last_clean":"2024-08-15T16:59:42.419430+0000","last_became_active":"2024-08-15T16:59:42.416665+0000","last_became_peered":"2024-08-15T16:59:42.416665+0000","last_unstale":"2024-08-15T16:59:42.419430+0000","last_undegraded":"2024-08-15T16:59:42.419430+0000","last_fullsized":"2024-08-15T16:59:42.419430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:31:42.855850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401340+0000","last_change":"2024-08-15T16:59:42.399470+0000","last_active":"2024-08-15T16:59:42.401340+0000","last_peered":"2024-08-15T16:59:42.401340+0000","last_clean":"2024-08-15T16:59:42.401340+0000","last_became_active":"2024-08-15T16:59:42.398232+0000","last_became_peered":"2024-08-15T16:59:42.398232+0000","last_unstale":"2024-08-15T16:59:42.401340+0000","last_undegraded":"2024-08-15T16:59:42.401340+0000","last_fullsized":"2024-08-15T16:59:42.401340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401337+0000","last_change":"2024-08-15T16:59:42.399759+0000","last_active":"2024-08-15T16:59:42.401337+0000","last_peered":"2024-08-15T16:59:42.401337+0000","last_clean":"2024-08-15T16:59:42.401337+0000","last_became_active":"2024-08-15T16:59:42.398288+0000","last_became_peered":"2024-08-15T16:59:42.398288+0000","last_unstale":"2024-08-15T16:59:42.401337+0000","last_undegraded":"2024-08-15T16:59:42.401337+0000","last_fullsized":"2024-08-15T16:59:42.401337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.402316+0000","last_change":"2024-08-15T16:59:42.397942+0000","last_active":"2024-08-15T16:59:42.402316+0000","last_peered":"2024-08-15T16:59:42.402316+0000","last_clean":"2024-08-15T16:59:42.402316+0000","last_became_active":"2024-08-15T16:59:42.397617+0000","last_became_peered":"2024-08-15T16:59:42.397617+0000","last_unstale":"2024-08-15T16:59:42.402316+0000","last_undegraded":"2024-08-15T16:59:42.402316+0000","last_fullsized":"2024-08-15T16:59:42.402316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:48:39.070398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416917+0000","last_change":"2024-08-15T16:59:42.408020+0000","last_active":"2024-08-15T16:59:42.416917+0000","last_peered":"2024-08-15T16:59:42.416917+0000","last_clean":"2024-08-15T16:59:42.416917+0000","last_became_active":"2024-08-15T16:59:42.407730+0000","last_became_peered":"2024-08-15T16:59:42.407730+0000","last_unstale":"2024-08-15T16:59:42.416917+0000","last_undegraded":"2024-08-15T16:59:42.416917+0000","last_fullsized":"2024-08-15T16:59:42.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420317+0000","last_change":"2024-08-15T16:59:42.415216+0000","last_active":"2024-08-15T16:59:42.420317+0000","last_peered":"2024-08-15T16:59:42.420317+0000","last_clean":"2024-08-15T16:59:42.420317+0000","last_became_active":"2024-08-15T16:59:42.414950+0000","last_became_peered":"2024-08-15T16:59:42.414950+0000","last_unstale":"2024-08-15T16:59:42.420317+0000","last_undegraded":"2024-08-15T16:59:42.420317+0000","last_fullsized":"2024-08-15T16:59:42.420317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:47:13.915180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411187+0000","last_change":"2024-08-15T16:59:42.403579+0000","last_active":"2024-08-15T16:59:42.411187+0000","last_peered":"2024-08-15T16:59:42.411187+0000","last_clean":"2024-08-15T16:59:42.411187+0000","last_became_active":"2024-08-15T16:59:42.403244+0000","last_became_peered":"2024-08-15T16:59:42.403244+0000","last_unstale":"2024-08-15T16:59:42.411187+0000","last_undegraded":"2024-08-15T16:59:42.411187+0000","last_fullsized":"2024-08-15T16:59:42.411187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414413+0000","last_change":"2024-08-15T16:59:42.407583+0000","last_active":"2024-08-15T16:59:42.414413+0000","last_peered":"2024-08-15T16:59:42.414413+0000","last_clean":"2024-08-15T16:59:42.414413+0000","last_became_active":"2024-08-15T16:59:42.406689+0000","last_became_peered":"2024-08-15T16:59:42.406689+0000","last_unstale":"2024-08-15T16:59:42.414413+0000","last_undegraded":"2024-08-15T16:59:42.414413+0000","last_fullsized":"2024-08-15T16:59:42.414413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:49:48.769242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414426+0000","last_change":"2024-08-15T16:59:42.404508+0000","last_active":"2024-08-15T16:59:42.414426+0000","last_peered":"2024-08-15T16:59:42.414426+0000","last_clean":"2024-08-15T16:59:42.414426+0000","last_became_active":"2024-08-15T16:59:42.404223+0000","last_became_peered":"2024-08-15T16:59:42.404223+0000","last_unstale":"2024-08-15T16:59:42.414426+0000","last_undegraded":"2024-08-15T16:59:42.414426+0000","last_fullsized":"2024-08-15T16:59:42.414426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:07:35.144380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.403776+0000","last_change":"2024-08-15T16:59:42.400163+0000","last_active":"2024-08-15T16:59:42.403776+0000","last_peered":"2024-08-15T16:59:42.403776+0000","last_clean":"2024-08-15T16:59:42.403776+0000","last_became_active":"2024-08-15T16:59:42.399738+0000","last_became_peered":"2024-08-15T16:59:42.399738+0000","last_unstale":"2024-08-15T16:59:42.403776+0000","last_undegraded":"2024-08-15T16:59:42.403776+0000","last_fullsized":"2024-08-15T16:59:42.403776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:40.046084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413534+0000","last_change":"2024-08-15T16:59:42.403318+0000","last_active":"2024-08-15T16:59:42.413534+0000","last_peered":"2024-08-15T16:59:42.413534+0000","last_clean":"2024-08-15T16:59:42.413534+0000","last_became_active":"2024-08-15T16:59:42.403051+0000","last_became_peered":"2024-08-15T16:59:42.403051+0000","last_unstale":"2024-08-15T16:59:42.413534+0000","last_undegraded":"2024-08-15T16:59:42.413534+0000","last_fullsized":"2024-08-15T16:59:42.413534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401457+0000","last_change":"2024-08-15T16:59:42.397107+0000","last_active":"2024-08-15T16:59:42.401457+0000","last_peered":"2024-08-15T16:59:42.401457+0000","last_clean":"2024-08-15T16:59:42.401457+0000","last_became_active":"2024-08-15T16:59:42.396786+0000","last_became_peered":"2024-08-15T16:59:42.396786+0000","last_unstale":"2024-08-15T16:59:42.401457+0000","last_undegraded":"2024-08-15T16:59:42.401457+0000","last_fullsized":"2024-08-15T16:59:42.401457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422555+0000","last_change":"2024-08-15T16:59:42.418246+0000","last_active":"2024-08-15T16:59:42.422555+0000","last_peered":"2024-08-15T16:59:42.422555+0000","last_clean":"2024-08-15T16:59:42.422555+0000","last_became_active":"2024-08-15T16:59:42.417975+0000","last_became_peered":"2024-08-15T16:59:42.417975+0000","last_unstale":"2024-08-15T16:59:42.422555+0000","last_undegraded":"2024-08-15T16:59:42.422555+0000","last_fullsized":"2024-08-15T16:59:42.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:46:29.993181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414618+0000","last_change":"2024-08-15T16:59:42.405054+0000","last_active":"2024-08-15T16:59:42.414618+0000","last_peered":"2024-08-15T16:59:42.414618+0000","last_clean":"2024-08-15T16:59:42.414618+0000","last_became_active":"2024-08-15T16:59:42.404788+0000","last_became_peered":"2024-08-15T16:59:42.404788+0000","last_unstale":"2024-08-15T16:59:42.414618+0000","last_undegraded":"2024-08-15T16:59:42.414618+0000","last_fullsized":"2024-08-15T16:59:42.414618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:49:25.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417014+0000","last_change":"2024-08-15T16:59:42.408237+0000","last_active":"2024-08-15T16:59:42.417014+0000","last_peered":"2024-08-15T16:59:42.417014+0000","last_clean":"2024-08-15T16:59:42.417014+0000","last_became_active":"2024-08-15T16:59:42.407321+0000","last_became_peered":"2024-08-15T16:59:42.407321+0000","last_unstale":"2024-08-15T16:59:42.417014+0000","last_undegraded":"2024-08-15T16:59:42.417014+0000","last_fullsized":"2024-08-15T16:59:42.417014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414394+0000","last_change":"2024-08-15T16:59:42.406550+0000","last_active":"2024-08-15T16:59:42.414394+0000","last_peered":"2024-08-15T16:59:42.414394+0000","last_clean":"2024-08-15T16:59:42.414394+0000","last_became_active":"2024-08-15T16:59:42.406280+0000","last_became_peered":"2024-08-15T16:59:42.406280+0000","last_unstale":"2024-08-15T16:59:42.414394+0000","last_undegraded":"2024-08-15T16:59:42.414394+0000","last_fullsized":"2024-08-15T16:59:42.414394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:47:40.336658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416946+0000","last_change":"2024-08-15T16:59:42.408293+0000","last_active":"2024-08-15T16:59:42.416946+0000","last_peered":"2024-08-15T16:59:42.416946+0000","last_clean":"2024-08-15T16:59:42.416946+0000","last_became_active":"2024-08-15T16:59:42.408005+0000","last_became_peered":"2024-08-15T16:59:42.408005+0000","last_unstale":"2024-08-15T16:59:42.416946+0000","last_undegraded":"2024-08-15T16:59:42.416946+0000","last_fullsized":"2024-08-15T16:59:42.416946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.408666+0000","last_change":"2024-08-15T16:59:42.403047+0000","last_active":"2024-08-15T16:59:42.408666+0000","last_peered":"2024-08-15T16:59:42.408666+0000","last_clean":"2024-08-15T16:59:42.408666+0000","last_became_active":"2024-08-15T16:59:42.402801+0000","last_became_peered":"2024-08-15T16:59:42.402801+0000","last_unstale":"2024-08-15T16:59:42.408666+0000","last_undegraded":"2024-08-15T16:59:42.408666+0000","last_fullsized":"2024-08-15T16:59:42.408666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416956+0000","last_change":"2024-08-15T16:59:42.411339+0000","last_active":"2024-08-15T16:59:42.416956+0000","last_peered":"2024-08-15T16:59:42.416956+0000","last_clean":"2024-08-15T16:59:42.416956+0000","last_became_active":"2024-08-15T16:59:42.411055+0000","last_became_peered":"2024-08-15T16:59:42.411055+0000","last_unstale":"2024-08-15T16:59:42.416956+0000","last_undegraded":"2024-08-15T16:59:42.416956+0000","last_fullsized":"2024-08-15T16:59:42.416956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417334+0000","last_change":"2024-08-15T16:59:42.411492+0000","last_active":"2024-08-15T16:59:42.417334+0000","last_peered":"2024-08-15T16:59:42.417334+0000","last_clean":"2024-08-15T16:59:42.417334+0000","last_became_active":"2024-08-15T16:59:42.411182+0000","last_became_peered":"2024-08-15T16:59:42.411182+0000","last_unstale":"2024-08-15T16:59:42.417334+0000","last_undegraded":"2024-08-15T16:59:42.417334+0000","last_fullsized":"2024-08-15T16:59:42.417334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420116+0000","last_change":"2024-08-15T16:59:42.412347+0000","last_active":"2024-08-15T16:59:42.420116+0000","last_peered":"2024-08-15T16:59:42.420116+0000","last_clean":"2024-08-15T16:59:42.420116+0000","last_became_active":"2024-08-15T16:59:42.411964+0000","last_became_peered":"2024-08-15T16:59:42.411964+0000","last_unstale":"2024-08-15T16:59:42.420116+0000","last_undegraded":"2024-08-15T16:59:42.420116+0000","last_fullsized":"2024-08-15T16:59:42.420116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:21.481110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414638+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.414638+0000","last_peered":"2024-08-15T16:59:42.414638+0000","last_clean":"2024-08-15T16:59:42.414638+0000","last_became_active":"2024-08-15T16:59:42.407063+0000","last_became_peered":"2024-08-15T16:59:42.407063+0000","last_unstale":"2024-08-15T16:59:42.414638+0000","last_undegraded":"2024-08-15T16:59:42.414638+0000","last_fullsized":"2024-08-15T16:59:42.414638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:54:52.191383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.405736+0000","last_change":"2024-08-15T16:59:42.398430+0000","last_active":"2024-08-15T16:59:42.405736+0000","last_peered":"2024-08-15T16:59:42.405736+0000","last_clean":"2024-08-15T16:59:42.405736+0000","last_became_active":"2024-08-15T16:59:42.397995+0000","last_became_peered":"2024-08-15T16:59:42.397995+0000","last_unstale":"2024-08-15T16:59:42.405736+0000","last_undegraded":"2024-08-15T16:59:42.405736+0000","last_fullsized":"2024-08-15T16:59:42.405736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:02.439971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418173+0000","last_change":"2024-08-15T16:59:42.417359+0000","last_active":"2024-08-15T16:59:42.418173+0000","last_peered":"2024-08-15T16:59:42.418173+0000","last_clean":"2024-08-15T16:59:42.418173+0000","last_became_active":"2024-08-15T16:59:42.416784+0000","last_became_peered":"2024-08-15T16:59:42.416784+0000","last_unstale":"2024-08-15T16:59:42.418173+0000","last_undegraded":"2024-08-15T16:59:42.418173+0000","last_fullsized":"2024-08-15T16:59:42.418173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400566+0000","last_change":"2024-08-15T16:59:42.396288+0000","last_active":"2024-08-15T16:59:42.400566+0000","last_peered":"2024-08-15T16:59:42.400566+0000","last_clean":"2024-08-15T16:59:42.400566+0000","last_became_active":"2024-08-15T16:59:42.395944+0000","last_became_peered":"2024-08-15T16:59:42.395944+0000","last_unstale":"2024-08-15T16:59:42.400566+0000","last_undegraded":"2024-08-15T16:59:42.400566+0000","last_fullsized":"2024-08-15T16:59:42.400566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419597+0000","last_change":"2024-08-15T16:59:41.381424+0000","last_active":"2024-08-15T16:59:42.419597+0000","last_peered":"2024-08-15T16:59:42.419597+0000","last_clean":"2024-08-15T16:59:42.419597+0000","last_became_active":"2024-08-15T16:59:41.381154+0000","last_became_peered":"2024-08-15T16:59:41.381154+0000","last_unstale":"2024-08-15T16:59:42.419597+0000","last_undegraded":"2024-08-15T16:59:42.419597+0000","last_fullsized":"2024-08-15T16:59:42.419597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420098+0000","last_change":"2024-08-15T16:59:42.412224+0000","last_active":"2024-08-15T16:59:42.420098+0000","last_peered":"2024-08-15T16:59:42.420098+0000","last_clean":"2024-08-15T16:59:42.420098+0000","last_became_active":"2024-08-15T16:59:42.411384+0000","last_became_peered":"2024-08-15T16:59:42.411384+0000","last_unstale":"2024-08-15T16:59:42.420098+0000","last_undegraded":"2024-08-15T16:59:42.420098+0000","last_fullsized":"2024-08-15T16:59:42.420098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:28:38.772967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.415389+0000","last_change":"2024-08-15T16:59:41.370657+0000","last_active":"2024-08-15T16:59:42.415389+0000","last_peered":"2024-08-15T16:59:42.415389+0000","last_clean":"2024-08-15T16:59:42.415389+0000","last_became_active":"2024-08-15T16:59:41.370303+0000","last_became_peered":"2024-08-15T16:59:41.370303+0000","last_unstale":"2024-08-15T16:59:42.415389+0000","last_undegraded":"2024-08-15T16:59:42.415389+0000","last_fullsized":"2024-08-15T16:59:42.415389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398754+0000","last_change":"2024-08-15T16:59:41.376440+0000","last_active":"2024-08-15T16:59:42.398754+0000","last_peered":"2024-08-15T16:59:42.398754+0000","last_clean":"2024-08-15T16:59:42.398754+0000","last_became_active":"2024-08-15T16:59:41.375356+0000","last_became_peered":"2024-08-15T16:59:41.375356+0000","last_unstale":"2024-08-15T16:59:42.398754+0000","last_undegraded":"2024-08-15T16:59:42.398754+0000","last_fullsized":"2024-08-15T16:59:42.398754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418845+0000","last_change":"2024-08-15T16:59:42.409055+0000","last_active":"2024-08-15T16:59:42.418845+0000","last_peered":"2024-08-15T16:59:42.418845+0000","last_clean":"2024-08-15T16:59:42.418845+0000","last_became_active":"2024-08-15T16:59:42.408704+0000","last_became_peered":"2024-08-15T16:59:42.408704+0000","last_unstale":"2024-08-15T16:59:42.418845+0000","last_undegraded":"2024-08-15T16:59:42.418845+0000","last_fullsized":"2024-08-15T16:59:42.418845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.388705+0000","last_change":"2024-08-15T16:59:41.372714+0000","last_active":"2024-08-15T16:59:42.388705+0000","last_peered":"2024-08-15T16:59:42.388705+0000","last_clean":"2024-08-15T16:59:42.388705+0000","last_became_active":"2024-08-15T16:59:41.372369+0000","last_became_peered":"2024-08-15T16:59:41.372369+0000","last_unstale":"2024-08-15T16:59:42.388705+0000","last_undegraded":"2024-08-15T16:59:42.388705+0000","last_fullsized":"2024-08-15T16:59:42.388705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:18.195005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417279+0000","last_change":"2024-08-15T16:59:42.412973+0000","last_active":"2024-08-15T16:59:42.417279+0000","last_peered":"2024-08-15T16:59:42.417279+0000","last_clean":"2024-08-15T16:59:42.417279+0000","last_became_active":"2024-08-15T16:59:42.412668+0000","last_became_peered":"2024-08-15T16:59:42.412668+0000","last_unstale":"2024-08-15T16:59:42.417279+0000","last_undegraded":"2024-08-15T16:59:42.417279+0000","last_fullsized":"2024-08-15T16:59:42.417279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.394506+0000","last_change":"2024-08-15T16:59:41.371198+0000","last_active":"2024-08-15T16:59:42.394506+0000","last_peered":"2024-08-15T16:59:42.394506+0000","last_clean":"2024-08-15T16:59:42.394506+0000","last_became_active":"2024-08-15T16:59:41.370260+0000","last_became_peered":"2024-08-15T16:59:41.370260+0000","last_unstale":"2024-08-15T16:59:42.394506+0000","last_undegraded":"2024-08-15T16:59:42.394506+0000","last_fullsized":"2024-08-15T16:59:42.394506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414362+0000","last_change":"2024-08-15T16:59:42.403938+0000","last_active":"2024-08-15T16:59:42.414362+0000","last_peered":"2024-08-15T16:59:42.414362+0000","last_clean":"2024-08-15T16:59:42.414362+0000","last_became_active":"2024-08-15T16:59:42.403693+0000","last_became_peered":"2024-08-15T16:59:42.403693+0000","last_unstale":"2024-08-15T16:59:42.414362+0000","last_undegraded":"2024-08-15T16:59:42.414362+0000","last_fullsized":"2024-08-15T16:59:42.414362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:46.202413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.390318+0000","last_change":"2024-08-15T16:59:41.370700+0000","last_active":"2024-08-15T16:59:42.390318+0000","last_peered":"2024-08-15T16:59:42.390318+0000","last_clean":"2024-08-15T16:59:42.390318+0000","last_became_active":"2024-08-15T16:59:41.370333+0000","last_became_peered":"2024-08-15T16:59:41.370333+0000","last_unstale":"2024-08-15T16:59:42.390318+0000","last_undegraded":"2024-08-15T16:59:42.390318+0000","last_fullsized":"2024-08-15T16:59:42.390318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:40.170510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420410+0000","last_change":"2024-08-15T16:59:42.415577+0000","last_active":"2024-08-15T16:59:42.420410+0000","last_peered":"2024-08-15T16:59:42.420410+0000","last_clean":"2024-08-15T16:59:42.420410+0000","last_became_active":"2024-08-15T16:59:42.415259+0000","last_became_peered":"2024-08-15T16:59:42.415259+0000","last_unstale":"2024-08-15T16:59:42.420410+0000","last_undegraded":"2024-08-15T16:59:42.420410+0000","last_fullsized":"2024-08-15T16:59:42.420410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:31.974939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420880+0000","last_change":"2024-08-15T16:59:41.382088+0000","last_active":"2024-08-15T16:59:42.420880+0000","last_peered":"2024-08-15T16:59:42.420880+0000","last_clean":"2024-08-15T16:59:42.420880+0000","last_became_active":"2024-08-15T16:59:41.380881+0000","last_became_peered":"2024-08-15T16:59:41.380881+0000","last_unstale":"2024-08-15T16:59:42.420880+0000","last_undegraded":"2024-08-15T16:59:42.420880+0000","last_fullsized":"2024-08-15T16:59:42.420880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.404994+0000","last_change":"2024-08-15T16:59:42.403887+0000","last_active":"2024-08-15T16:59:42.404994+0000","last_peered":"2024-08-15T16:59:42.404994+0000","last_clean":"2024-08-15T16:59:42.404994+0000","last_became_active":"2024-08-15T16:59:42.403619+0000","last_became_peered":"2024-08-15T16:59:42.403619+0000","last_unstale":"2024-08-15T16:59:42.404994+0000","last_undegraded":"2024-08-15T16:59:42.404994+0000","last_fullsized":"2024-08-15T16:59:42.404994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421139+0000","last_change":"2024-08-15T16:59:41.379757+0000","last_active":"2024-08-15T16:59:42.421139+0000","last_peered":"2024-08-15T16:59:42.421139+0000","last_clean":"2024-08-15T16:59:42.421139+0000","last_became_active":"2024-08-15T16:59:41.379393+0000","last_became_peered":"2024-08-15T16:59:41.379393+0000","last_unstale":"2024-08-15T16:59:42.421139+0000","last_undegraded":"2024-08-15T16:59:42.421139+0000","last_fullsized":"2024-08-15T16:59:42.421139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418889+0000","last_change":"2024-08-15T16:59:42.413445+0000","last_active":"2024-08-15T16:59:42.418889+0000","last_peered":"2024-08-15T16:59:42.418889+0000","last_clean":"2024-08-15T16:59:42.418889+0000","last_became_active":"2024-08-15T16:59:42.413176+0000","last_became_peered":"2024-08-15T16:59:42.413176+0000","last_unstale":"2024-08-15T16:59:42.418889+0000","last_undegraded":"2024-08-15T16:59:42.418889+0000","last_fullsized":"2024-08-15T16:59:42.418889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420263+0000","last_change":"2024-08-15T16:59:41.381453+0000","last_active":"2024-08-15T16:59:42.420263+0000","last_peered":"2024-08-15T16:59:42.420263+0000","last_clean":"2024-08-15T16:59:42.420263+0000","last_became_active":"2024-08-15T16:59:41.381090+0000","last_became_peered":"2024-08-15T16:59:41.381090+0000","last_unstale":"2024-08-15T16:59:42.420263+0000","last_undegraded":"2024-08-15T16:59:42.420263+0000","last_fullsized":"2024-08-15T16:59:42.420263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420060+0000","last_change":"2024-08-15T16:59:42.411131+0000","last_active":"2024-08-15T16:59:42.420060+0000","last_peered":"2024-08-15T16:59:42.420060+0000","last_clean":"2024-08-15T16:59:42.420060+0000","last_became_active":"2024-08-15T16:59:42.410756+0000","last_became_peered":"2024-08-15T16:59:42.410756+0000","last_unstale":"2024-08-15T16:59:42.420060+0000","last_undegraded":"2024-08-15T16:59:42.420060+0000","last_fullsized":"2024-08-15T16:59:42.420060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391610+0000","last_change":"2024-08-15T16:59:41.373319+0000","last_active":"2024-08-15T16:59:42.391610+0000","last_peered":"2024-08-15T16:59:42.391610+0000","last_clean":"2024-08-15T16:59:42.391610+0000","last_became_active":"2024-08-15T16:59:41.369550+0000","last_became_peered":"2024-08-15T16:59:41.369550+0000","last_unstale":"2024-08-15T16:59:42.391610+0000","last_undegraded":"2024-08-15T16:59:42.391610+0000","last_fullsized":"2024-08-15T16:59:42.391610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:16:34.424937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413118+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.413118+0000","last_peered":"2024-08-15T16:59:42.413118+0000","last_clean":"2024-08-15T16:59:42.413118+0000","last_became_active":"2024-08-15T16:59:42.407033+0000","last_became_peered":"2024-08-15T16:59:42.407033+0000","last_unstale":"2024-08-15T16:59:42.413118+0000","last_undegraded":"2024-08-15T16:59:42.413118+0000","last_fullsized":"2024-08-15T16:59:42.413118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:58.883271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.392955+0000","last_change":"2024-08-15T16:59:41.370530+0000","last_active":"2024-08-15T16:59:42.392955+0000","last_peered":"2024-08-15T16:59:42.392955+0000","last_clean":"2024-08-15T16:59:42.392955+0000","last_became_active":"2024-08-15T16:59:41.369807+0000","last_became_peered":"2024-08-15T16:59:41.369807+0000","last_unstale":"2024-08-15T16:59:42.392955+0000","last_undegraded":"2024-08-15T16:59:42.392955+0000","last_fullsized":"2024-08-15T16:59:42.392955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418194+0000","last_change":"2024-08-15T16:59:42.417158+0000","last_active":"2024-08-15T16:59:42.418194+0000","last_peered":"2024-08-15T16:59:42.418194+0000","last_clean":"2024-08-15T16:59:42.418194+0000","last_became_active":"2024-08-15T16:59:42.416838+0000","last_became_peered":"2024-08-15T16:59:42.416838+0000","last_unstale":"2024-08-15T16:59:42.418194+0000","last_undegraded":"2024-08-15T16:59:42.418194+0000","last_fullsized":"2024-08-15T16:59:42.418194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422895+0000","last_change":"2024-08-15T16:59:41.377962+0000","last_active":"2024-08-15T16:59:42.422895+0000","last_peered":"2024-08-15T16:59:42.422895+0000","last_clean":"2024-08-15T16:59:42.422895+0000","last_became_active":"2024-08-15T16:59:41.377699+0000","last_became_peered":"2024-08-15T16:59:41.377699+0000","last_unstale":"2024-08-15T16:59:42.422895+0000","last_undegraded":"2024-08-15T16:59:42.422895+0000","last_fullsized":"2024-08-15T16:59:42.422895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413284+0000","last_change":"2024-08-15T16:59:42.407316+0000","last_active":"2024-08-15T16:59:42.413284+0000","last_peered":"2024-08-15T16:59:42.413284+0000","last_clean":"2024-08-15T16:59:42.413284+0000","last_became_active":"2024-08-15T16:59:42.407049+0000","last_became_peered":"2024-08-15T16:59:42.407049+0000","last_unstale":"2024-08-15T16:59:42.413284+0000","last_undegraded":"2024-08-15T16:59:42.413284+0000","last_fullsized":"2024-08-15T16:59:42.413284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418920+0000","last_change":"2024-08-15T16:59:41.373342+0000","last_active":"2024-08-15T16:59:42.418920+0000","last_peered":"2024-08-15T16:59:42.418920+0000","last_clean":"2024-08-15T16:59:42.418920+0000","last_became_active":"2024-08-15T16:59:41.373010+0000","last_became_peered":"2024-08-15T16:59:41.373010+0000","last_unstale":"2024-08-15T16:59:42.418920+0000","last_undegraded":"2024-08-15T16:59:42.418920+0000","last_fullsized":"2024-08-15T16:59:42.418920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413438+0000","last_change":"2024-08-15T16:59:42.404605+0000","last_active":"2024-08-15T16:59:42.413438+0000","last_peered":"2024-08-15T16:59:42.413438+0000","last_clean":"2024-08-15T16:59:42.413438+0000","last_became_active":"2024-08-15T16:59:42.404054+0000","last_became_peered":"2024-08-15T16:59:42.404054+0000","last_unstale":"2024-08-15T16:59:42.413438+0000","last_undegraded":"2024-08-15T16:59:42.413438+0000","last_fullsized":"2024-08-15T16:59:42.413438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419684+0000","last_change":"2024-08-15T16:59:41.381959+0000","last_active":"2024-08-15T16:59:42.419684+0000","last_peered":"2024-08-15T16:59:42.419684+0000","last_clean":"2024-08-15T16:59:42.419684+0000","last_became_active":"2024-08-15T16:59:41.381634+0000","last_became_peered":"2024-08-15T16:59:41.381634+0000","last_unstale":"2024-08-15T16:59:42.419684+0000","last_undegraded":"2024-08-15T16:59:42.419684+0000","last_fullsized":"2024-08-15T16:59:42.419684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413316+0000","last_change":"2024-08-15T16:59:42.408082+0000","last_active":"2024-08-15T16:59:42.413316+0000","last_peered":"2024-08-15T16:59:42.413316+0000","last_clean":"2024-08-15T16:59:42.413316+0000","last_became_active":"2024-08-15T16:59:42.407840+0000","last_became_peered":"2024-08-15T16:59:42.407840+0000","last_unstale":"2024-08-15T16:59:42.413316+0000","last_undegraded":"2024-08-15T16:59:42.413316+0000","last_fullsized":"2024-08-15T16:59:42.413316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:32.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422292+0000","last_change":"2024-08-15T16:59:41.381232+0000","last_active":"2024-08-15T16:59:42.422292+0000","last_peered":"2024-08-15T16:59:42.422292+0000","last_clean":"2024-08-15T16:59:42.422292+0000","last_became_active":"2024-08-15T16:59:41.380497+0000","last_became_peered":"2024-08-15T16:59:41.380497+0000","last_unstale":"2024-08-15T16:59:42.422292+0000","last_undegraded":"2024-08-15T16:59:42.422292+0000","last_fullsized":"2024-08-15T16:59:42.422292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414529+0000","last_change":"2024-08-15T16:59:42.410719+0000","last_active":"2024-08-15T16:59:42.414529+0000","last_peered":"2024-08-15T16:59:42.414529+0000","last_clean":"2024-08-15T16:59:42.414529+0000","last_became_active":"2024-08-15T16:59:42.410323+0000","last_became_peered":"2024-08-15T16:59:42.410323+0000","last_unstale":"2024-08-15T16:59:42.414529+0000","last_undegraded":"2024-08-15T16:59:42.414529+0000","last_fullsized":"2024-08-15T16:59:42.414529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:22:43.534462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387934+0000","last_change":"2024-08-15T16:59:41.375212+0000","last_active":"2024-08-15T16:59:42.387934+0000","last_peered":"2024-08-15T16:59:42.387934+0000","last_clean":"2024-08-15T16:59:42.387934+0000","last_became_active":"2024-08-15T16:59:41.374811+0000","last_became_peered":"2024-08-15T16:59:41.374811+0000","last_unstale":"2024-08-15T16:59:42.387934+0000","last_undegraded":"2024-08-15T16:59:42.387934+0000","last_fullsized":"2024-08-15T16:59:42.387934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419483+0000","last_change":"2024-08-15T16:59:42.416559+0000","last_active":"2024-08-15T16:59:42.419483+0000","last_peered":"2024-08-15T16:59:42.419483+0000","last_clean":"2024-08-15T16:59:42.419483+0000","last_became_active":"2024-08-15T16:59:42.416241+0000","last_became_peered":"2024-08-15T16:59:42.416241+0000","last_unstale":"2024-08-15T16:59:42.419483+0000","last_undegraded":"2024-08-15T16:59:42.419483+0000","last_fullsized":"2024-08-15T16:59:42.419483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:30:00.824354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395661+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:42.395661+0000","last_peered":"2024-08-15T16:59:42.395661+0000","last_clean":"2024-08-15T16:59:42.395661+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:42.395661+0000","last_undegraded":"2024-08-15T16:59:42.395661+0000","last_fullsized":"2024-08-15T16:59:42.395661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.386944+0000","last_change":"2024-08-15T16:59:41.374273+0000","last_active":"2024-08-15T16:59:42.386944+0000","last_peered":"2024-08-15T16:59:42.386944+0000","last_clean":"2024-08-15T16:59:42.386944+0000","last_became_active":"2024-08-15T16:59:41.373947+0000","last_became_peered":"2024-08-15T16:59:41.373947+0000","last_unstale":"2024-08-15T16:59:42.386944+0000","last_undegraded":"2024-08-15T16:59:42.386944+0000","last_fullsized":"2024-08-15T16:59:42.386944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:59:06.484884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417638+0000","last_change":"2024-08-15T16:59:42.412036+0000","last_active":"2024-08-15T16:59:42.417638+0000","last_peered":"2024-08-15T16:59:42.417638+0000","last_clean":"2024-08-15T16:59:42.417638+0000","last_became_active":"2024-08-15T16:59:42.411769+0000","last_became_peered":"2024-08-15T16:59:42.411769+0000","last_unstale":"2024-08-15T16:59:42.417638+0000","last_undegraded":"2024-08-15T16:59:42.417638+0000","last_fullsized":"2024-08-15T16:59:42.417638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395564+0000","last_change":"2024-08-15T16:59:41.384227+0000","last_active":"2024-08-15T16:59:42.395564+0000","last_peered":"2024-08-15T16:59:42.395564+0000","last_clean":"2024-08-15T16:59:42.395564+0000","last_became_active":"2024-08-15T16:59:41.384013+0000","last_became_peered":"2024-08-15T16:59:41.384013+0000","last_unstale":"2024-08-15T16:59:42.395564+0000","last_undegraded":"2024-08-15T16:59:42.395564+0000","last_fullsized":"2024-08-15T16:59:42.395564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417479+0000","last_change":"2024-08-15T16:59:42.406032+0000","last_active":"2024-08-15T16:59:42.417479+0000","last_peered":"2024-08-15T16:59:42.417479+0000","last_clean":"2024-08-15T16:59:42.417479+0000","last_became_active":"2024-08-15T16:59:42.405742+0000","last_became_peered":"2024-08-15T16:59:42.405742+0000","last_unstale":"2024-08-15T16:59:42.417479+0000","last_undegraded":"2024-08-15T16:59:42.417479+0000","last_fullsized":"2024-08-15T16:59:42.417479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387905+0000","last_change":"2024-08-15T16:59:41.382128+0000","last_active":"2024-08-15T16:59:42.387905+0000","last_peered":"2024-08-15T16:59:42.387905+0000","last_clean":"2024-08-15T16:59:42.387905+0000","last_became_active":"2024-08-15T16:59:41.381700+0000","last_became_peered":"2024-08-15T16:59:41.381700+0000","last_unstale":"2024-08-15T16:59:42.387905+0000","last_undegraded":"2024-08-15T16:59:42.387905+0000","last_fullsized":"2024-08-15T16:59:42.387905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414769+0000","last_change":"2024-08-15T16:59:42.407654+0000","last_active":"2024-08-15T16:59:42.414769+0000","last_peered":"2024-08-15T16:59:42.414769+0000","last_clean":"2024-08-15T16:59:42.414769+0000","last_became_active":"2024-08-15T16:59:42.405202+0000","last_became_peered":"2024-08-15T16:59:42.405202+0000","last_unstale":"2024-08-15T16:59:42.414769+0000","last_undegraded":"2024-08-15T16:59:42.414769+0000","last_fullsized":"2024-08-15T16:59:42.414769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391657+0000","last_change":"2024-08-15T16:59:41.374124+0000","last_active":"2024-08-15T16:59:42.391657+0000","last_peered":"2024-08-15T16:59:42.391657+0000","last_clean":"2024-08-15T16:59:42.391657+0000","last_became_active":"2024-08-15T16:59:41.373620+0000","last_became_peered":"2024-08-15T16:59:41.373620+0000","last_unstale":"2024-08-15T16:59:42.391657+0000","last_undegraded":"2024-08-15T16:59:42.391657+0000","last_fullsized":"2024-08-15T16:59:42.391657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:55:35.560885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.410018+0000","last_change":"2024-08-15T16:59:42.408577+0000","last_active":"2024-08-15T16:59:42.410018+0000","last_peered":"2024-08-15T16:59:42.410018+0000","last_clean":"2024-08-15T16:59:42.410018+0000","last_became_active":"2024-08-15T16:59:42.408272+0000","last_became_peered":"2024-08-15T16:59:42.408272+0000","last_unstale":"2024-08-15T16:59:42.410018+0000","last_undegraded":"2024-08-15T16:59:42.410018+0000","last_fullsized":"2024-08-15T16:59:42.410018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:55.794089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.396515+0000","last_change":"2024-08-15T16:59:41.383124+0000","last_active":"2024-08-15T16:59:42.396515+0000","last_peered":"2024-08-15T16:59:42.396515+0000","last_clean":"2024-08-15T16:59:42.396515+0000","last_became_active":"2024-08-15T16:59:41.382696+0000","last_became_peered":"2024-08-15T16:59:41.382696+0000","last_unstale":"2024-08-15T16:59:42.396515+0000","last_undegraded":"2024-08-15T16:59:42.396515+0000","last_fullsized":"2024-08-15T16:59:42.396515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414332+0000","last_change":"2024-08-15T16:59:42.406595+0000","last_active":"2024-08-15T16:59:42.414332+0000","last_peered":"2024-08-15T16:59:42.414332+0000","last_clean":"2024-08-15T16:59:42.414332+0000","last_became_active":"2024-08-15T16:59:42.406353+0000","last_became_peered":"2024-08-15T16:59:42.406353+0000","last_unstale":"2024-08-15T16:59:42.414332+0000","last_undegraded":"2024-08-15T16:59:42.414332+0000","last_fullsized":"2024-08-15T16:59:42.414332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:25:43.883310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398385+0000","last_change":"2024-08-15T16:59:41.381997+0000","last_active":"2024-08-15T16:59:42.398385+0000","last_peered":"2024-08-15T16:59:42.398385+0000","last_clean":"2024-08-15T16:59:42.398385+0000","last_became_active":"2024-08-15T16:59:41.381679+0000","last_became_peered":"2024-08-15T16:59:41.381679+0000","last_unstale":"2024-08-15T16:59:42.398385+0000","last_undegraded":"2024-08-15T16:59:42.398385+0000","last_fullsized":"2024-08-15T16:59:42.398385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:39.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411157+0000","last_change":"2024-08-15T16:59:42.404468+0000","last_active":"2024-08-15T16:59:42.411157+0000","last_peered":"2024-08-15T16:59:42.411157+0000","last_clean":"2024-08-15T16:59:42.411157+0000","last_became_active":"2024-08-15T16:59:42.403172+0000","last_became_peered":"2024-08-15T16:59:42.403172+0000","last_unstale":"2024-08-15T16:59:42.411157+0000","last_undegraded":"2024-08-15T16:59:42.411157+0000","last_fullsized":"2024-08-15T16:59:42.411157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T16:59:43.395993+0000","last_change":"2024-08-15T16:59:41.379177+0000","last_active":"2024-08-15T16:59:43.395993+0000","last_peered":"2024-08-15T16:59:43.395993+0000","last_clean":"2024-08-15T16:59:43.395993+0000","last_became_active":"2024-08-15T16:59:41.378909+0000","last_became_peered":"2024-08-15T16:59:41.378909+0000","last_unstale":"2024-08-15T16:59:43.395993+0000","last_undegraded":"2024-08-15T16:59:43.395993+0000","last_fullsized":"2024-08-15T16:59:43.395993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416728+0000","last_change":"2024-08-15T16:59:42.407782+0000","last_active":"2024-08-15T16:59:42.416728+0000","last_peered":"2024-08-15T16:59:42.416728+0000","last_clean":"2024-08-15T16:59:42.416728+0000","last_became_active":"2024-08-15T16:59:42.407483+0000","last_became_peered":"2024-08-15T16:59:42.407483+0000","last_unstale":"2024-08-15T16:59:42.416728+0000","last_undegraded":"2024-08-15T16:59:42.416728+0000","last_fullsized":"2024-08-15T16:59:42.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420095+0000","last_change":"2024-08-15T16:59:41.379440+0000","last_active":"2024-08-15T16:59:42.420095+0000","last_peered":"2024-08-15T16:59:42.420095+0000","last_clean":"2024-08-15T16:59:42.420095+0000","last_became_active":"2024-08-15T16:59:41.379116+0000","last_became_peered":"2024-08-15T16:59:41.379116+0000","last_unstale":"2024-08-15T16:59:42.420095+0000","last_undegraded":"2024-08-15T16:59:42.420095+0000","last_fullsized":"2024-08-15T16:59:42.420095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420125+0000","last_change":"2024-08-15T16:59:42.414998+0000","last_active":"2024-08-15T16:59:42.420125+0000","last_peered":"2024-08-15T16:59:42.420125+0000","last_clean":"2024-08-15T16:59:42.420125+0000","last_became_active":"2024-08-15T16:59:42.414696+0000","last_became_peered":"2024-08-15T16:59:42.414696+0000","last_unstale":"2024-08-15T16:59:42.420125+0000","last_undegraded":"2024-08-15T16:59:42.420125+0000","last_fullsized":"2024-08-15T16:59:42.420125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:42:39.871776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395563+0000","last_change":"2024-08-15T16:59:41.381044+0000","last_active":"2024-08-15T16:59:42.395563+0000","last_peered":"2024-08-15T16:59:42.395563+0000","last_clean":"2024-08-15T16:59:42.395563+0000","last_became_active":"2024-08-15T16:59:41.380776+0000","last_became_peered":"2024-08-15T16:59:41.380776+0000","last_unstale":"2024-08-15T16:59:42.395563+0000","last_undegraded":"2024-08-15T16:59:42.395563+0000","last_fullsized":"2024-08-15T16:59:42.395563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:21:56.894684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.409831+0000","last_change":"2024-08-15T16:59:42.408779+0000","last_active":"2024-08-15T16:59:42.409831+0000","last_peered":"2024-08-15T16:59:42.409831+0000","last_clean":"2024-08-15T16:59:42.409831+0000","last_became_active":"2024-08-15T16:59:42.408345+0000","last_became_peered":"2024-08-15T16:59:42.408345+0000","last_unstale":"2024-08-15T16:59:42.409831+0000","last_undegraded":"2024-08-15T16:59:42.409831+0000","last_fullsized":"2024-08-15T16:59:42.409831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:55:03.519854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395271+0000","last_change":"2024-08-15T16:59:41.382297+0000","last_active":"2024-08-15T16:59:42.395271+0000","last_peered":"2024-08-15T16:59:42.395271+0000","last_clean":"2024-08-15T16:59:42.395271+0000","last_became_active":"2024-08-15T16:59:41.381812+0000","last_became_peered":"2024-08-15T16:59:41.381812+0000","last_unstale":"2024-08-15T16:59:42.395271+0000","last_undegraded":"2024-08-15T16:59:42.395271+0000","last_fullsized":"2024-08-15T16:59:42.395271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418823+0000","last_change":"2024-08-15T16:59:42.402821+0000","last_active":"2024-08-15T16:59:42.418823+0000","last_peered":"2024-08-15T16:59:42.418823+0000","last_clean":"2024-08-15T16:59:42.418823+0000","last_became_active":"2024-08-15T16:59:42.402044+0000","last_became_peered":"2024-08-15T16:59:42.402044+0000","last_unstale":"2024-08-15T16:59:42.418823+0000","last_undegraded":"2024-08-15T16:59:42.418823+0000","last_fullsized":"2024-08-15T16:59:42.418823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395855+0000","last_change":"2024-08-15T16:59:41.383793+0000","last_active":"2024-08-15T16:59:42.395855+0000","last_peered":"2024-08-15T16:59:42.395855+0000","last_clean":"2024-08-15T16:59:42.395855+0000","last_became_active":"2024-08-15T16:59:41.383431+0000","last_became_peered":"2024-08-15T16:59:41.383431+0000","last_unstale":"2024-08-15T16:59:42.395855+0000","last_undegraded":"2024-08-15T16:59:42.395855+0000","last_fullsized":"2024-08-15T16:59:42.395855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422596+0000","last_change":"2024-08-15T16:59:42.419454+0000","last_active":"2024-08-15T16:59:42.422596+0000","last_peered":"2024-08-15T16:59:42.422596+0000","last_clean":"2024-08-15T16:59:42.422596+0000","last_became_active":"2024-08-15T16:59:42.419157+0000","last_became_peered":"2024-08-15T16:59:42.419157+0000","last_unstale":"2024-08-15T16:59:42.422596+0000","last_undegraded":"2024-08-15T16:59:42.422596+0000","last_fullsized":"2024-08-15T16:59:42.422596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421197+0000","last_change":"2024-08-15T16:59:41.382836+0000","last_active":"2024-08-15T16:59:42.421197+0000","last_peered":"2024-08-15T16:59:42.421197+0000","last_clean":"2024-08-15T16:59:42.421197+0000","last_became_active":"2024-08-15T16:59:41.382595+0000","last_became_peered":"2024-08-15T16:59:41.382595+0000","last_unstale":"2024-08-15T16:59:42.421197+0000","last_undegraded":"2024-08-15T16:59:42.421197+0000","last_fullsized":"2024-08-15T16:59:42.421197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401292+0000","last_change":"2024-08-15T16:59:42.399359+0000","last_active":"2024-08-15T16:59:42.401292+0000","last_peered":"2024-08-15T16:59:42.401292+0000","last_clean":"2024-08-15T16:59:42.401292+0000","last_became_active":"2024-08-15T16:59:42.398800+0000","last_became_peered":"2024-08-15T16:59:42.398800+0000","last_unstale":"2024-08-15T16:59:42.401292+0000","last_undegraded":"2024-08-15T16:59:42.401292+0000","last_fullsized":"2024-08-15T16:59:42.401292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.397841+0000","last_change":"2024-08-15T16:59:42.396158+0000","last_active":"2024-08-15T16:59:42.397841+0000","last_peered":"2024-08-15T16:59:42.397841+0000","last_clean":"2024-08-15T16:59:42.397841+0000","last_became_active":"2024-08-15T16:59:42.395850+0000","last_became_peered":"2024-08-15T16:59:42.395850+0000","last_unstale":"2024-08-15T16:59:42.397841+0000","last_undegraded":"2024-08-15T16:59:42.397841+0000","last_fullsized":"2024-08-15T16:59:42.397841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:39.678214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422026+0000","last_change":"2024-08-15T16:59:41.385558+0000","last_active":"2024-08-15T16:59:42.422026+0000","last_peered":"2024-08-15T16:59:42.422026+0000","last_clean":"2024-08-15T16:59:42.422026+0000","last_became_active":"2024-08-15T16:59:41.385303+0000","last_became_peered":"2024-08-15T16:59:41.385303+0000","last_unstale":"2024-08-15T16:59:42.422026+0000","last_undegraded":"2024-08-15T16:59:42.422026+0000","last_fullsized":"2024-08-15T16:59:42.422026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400920+0000","last_change":"2024-08-15T16:59:42.396765+0000","last_active":"2024-08-15T16:59:42.400920+0000","last_peered":"2024-08-15T16:59:42.400920+0000","last_clean":"2024-08-15T16:59:42.400920+0000","last_became_active":"2024-08-15T16:59:42.396133+0000","last_became_peered":"2024-08-15T16:59:42.396133+0000","last_unstale":"2024-08-15T16:59:42.400920+0000","last_undegraded":"2024-08-15T16:59:42.400920+0000","last_fullsized":"2024-08-15T16:59:42.400920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:38:03.347839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420319+0000","last_change":"2024-08-15T16:59:41.381180+0000","last_active":"2024-08-15T16:59:42.420319+0000","last_peered":"2024-08-15T16:59:42.420319+0000","last_clean":"2024-08-15T16:59:42.420319+0000","last_became_active":"2024-08-15T16:59:41.381011+0000","last_became_peered":"2024-08-15T16:59:41.381011+0000","last_unstale":"2024-08-15T16:59:42.420319+0000","last_undegraded":"2024-08-15T16:59:42.420319+0000","last_fullsized":"2024-08-15T16:59:42.420319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420390+0000","last_change":"2024-08-15T16:59:42.416366+0000","last_active":"2024-08-15T16:59:42.420390+0000","last_peered":"2024-08-15T16:59:42.420390+0000","last_clean":"2024-08-15T16:59:42.420390+0000","last_became_active":"2024-08-15T16:59:42.415630+0000","last_became_peered":"2024-08-15T16:59:42.415630+0000","last_unstale":"2024-08-15T16:59:42.420390+0000","last_undegraded":"2024-08-15T16:59:42.420390+0000","last_fullsized":"2024-08-15T16:59:42.420390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:23.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420472+0000","last_change":"2024-08-15T16:59:41.379215+0000","last_active":"2024-08-15T16:59:42.420472+0000","last_peered":"2024-08-15T16:59:42.420472+0000","last_clean":"2024-08-15T16:59:42.420472+0000","last_became_active":"2024-08-15T16:59:41.378905+0000","last_became_peered":"2024-08-15T16:59:41.378905+0000","last_unstale":"2024-08-15T16:59:42.420472+0000","last_undegraded":"2024-08-15T16:59:42.420472+0000","last_fullsized":"2024-08-15T16:59:42.420472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416924+0000","last_change":"2024-08-15T16:59:42.408126+0000","last_active":"2024-08-15T16:59:42.416924+0000","last_peered":"2024-08-15T16:59:42.416924+0000","last_clean":"2024-08-15T16:59:42.416924+0000","last_became_active":"2024-08-15T16:59:42.407836+0000","last_became_peered":"2024-08-15T16:59:42.407836+0000","last_unstale":"2024-08-15T16:59:42.416924+0000","last_undegraded":"2024-08-15T16:59:42.416924+0000","last_fullsized":"2024-08-15T16:59:42.416924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421366+0000","last_change":"2024-08-15T16:59:41.382335+0000","last_active":"2024-08-15T16:59:42.421366+0000","last_peered":"2024-08-15T16:59:42.421366+0000","last_clean":"2024-08-15T16:59:42.421366+0000","last_became_active":"2024-08-15T16:59:41.382004+0000","last_became_peered":"2024-08-15T16:59:41.382004+0000","last_unstale":"2024-08-15T16:59:42.421366+0000","last_undegraded":"2024-08-15T16:59:42.421366+0000","last_fullsized":"2024-08-15T16:59:42.421366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":811008,"data_stored":664727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":217088,"data_stored":73879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":73915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":212992,"data_stored":73789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":663822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:52.198 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T16:59:52.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 Processor -- start 2024-08-15T16:59:52.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 -- start start 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 0x7f110c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f110c0aff40 0x7f110c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0a9f60 con 0x7f110c155380 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c06e4b0 con 0x7f110c0aff40 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1110d07700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c06e6a0 con 0x7f110c0af5a0 2024-08-15T16:59:52.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f110c0aff40 0x7f110c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f110c0aff40 0x7f110c155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49864/0 (socket says 172.21.15.146:49864) 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 -- 172.21.15.146:0/2234790155 learned_addr learned my addr 172.21.15.146:0/2234790155 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 -- 172.21.15.146:0/2234790155 >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 msgr2=0x7f110c0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1109d9b700 1 --2- 172.21.15.146:0/2234790155 >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 0x7f110c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 --2- 172.21.15.146:0/2234790155 >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 0x7f110c0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 -- 172.21.15.146:0/2234790155 >> v1:172.21.15.146:6789/0 conn(0x7f110c155380 legacy=0x7f110c0a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 -- 172.21.15.146:0/2234790155 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c06e980 con 0x7f110c0aff40 2024-08-15T16:59:52.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f1109d9b700 1 --2- 172.21.15.146:0/2234790155 >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 0x7f110c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110ad9d700 1 --2- 172.21.15.146:0/2234790155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c155750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f110c159240 tx=0x7f1104009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54f050d58702592 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110959a700 1 -- 172.21.15.146:0/2234790155 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040092e0 con 0x7f110c0aff40 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110959a700 1 -- 172.21.15.146:0/2234790155 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400ba50 con 0x7f110c0aff40 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.285+0000 7f110959a700 1 -- 172.21.15.146:0/2234790155 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400be10 con 0x7f110c0aff40 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- 172.21.15.146:0/2234790155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 msgr2=0x7f110c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- 172.21.15.146:0/2234790155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c155750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f110c159240 tx=0x7f1104009840 comp rx=0 tx=0).stop 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- 172.21.15.146:0/2234790155 shutdown_connections 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- 172.21.15.146:0/2234790155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c155750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- 172.21.15.146:0/2234790155 >> v2:172.21.15.161:3300/0 conn(0x7f110c0af5a0 0x7f110c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- 172.21.15.146:0/2234790155 >> 172.21.15.146:0/2234790155 conn(0x7f110c066e00 msgr2=0x7f110c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- 172.21.15.146:0/2234790155 shutdown_connections 2024-08-15T16:59:52.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- 172.21.15.146:0/2234790155 wait complete. 2024-08-15T16:59:52.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 Processor -- start 2024-08-15T16:59:52.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- start start 2024-08-15T16:59:52.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f110c0af5a0 0x7f110c14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f110c155380 0x7f110c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1577b0 con 0x7f110c0aff40 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c157a20 con 0x7f110c0af5a0 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1110d07700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c157d00 con 0x7f110c155380 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49874/0 (socket says 172.21.15.146:49874) 2024-08-15T16:59:52.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 -- 172.21.15.146:0/1183285022 learned_addr learned my addr 172.21.15.146:0/1183285022 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f110a59c700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f110c0af5a0 0x7f110c14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f110ad9d700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f110c155380 0x7f110c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 -- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f110c155380 msgr2=0x7f110c157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f110c155380 0x7f110c157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 -- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f110c0af5a0 msgr2=0x7f110c14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f110c0af5a0 0x7f110c14cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.289+0000 7f1109d9b700 1 -- 172.21.15.146:0/1183285022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104009ae0 con 0x7f110c0aff40 2024-08-15T16:59:52.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f1109d9b700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c850 tx=0x7f10f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4011600 con 0x7f110c0aff40 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c157fe0 con 0x7f110c0aff40 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4011760 con 0x7f110c0aff40 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4011b00 con 0x7f110c0aff40 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c169690 con 0x7f110c0aff40 2024-08-15T16:59:52.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c169910 con 0x7f110c0aff40 2024-08-15T16:59:52.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f10f40073b0 con 0x7f110c0aff40 2024-08-15T16:59:52.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 0x7f10f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f10f407cbd0 con 0x7f110c0aff40 2024-08-15T16:59:52.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f110a59c700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 0x7f10f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.293+0000 7f110a59c700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 0x7f10f0062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f110c0aa0f0 tx=0x7f10fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.301+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f110c169910 con 0x7f110c0aff40 2024-08-15T16:59:52.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.445+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f110c169910 con 0x7f10f0060860 2024-08-15T16:59:52.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.465+0000 7f11027fc700 1 -- 172.21.15.146:0/1183285022 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264493 (secure 0 0 0) 0x7f110c169910 con 0x7f10f0060860 2024-08-15T16:59:52.481 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 msgr2=0x7f10f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 0x7f10f0062d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f110c0aa0f0 tx=0x7f10fc009040 comp rx=0 tx=0).stop 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 msgr2=0x7f110c0c5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10f400c850 tx=0x7f10f4008040 comp rx=0 tx=0).stop 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 shutdown_connections 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10f0060860 0x7f10f0062d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f110c155380 0x7f110c157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f110c0aff40 0x7f110c0c5980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 --2- 172.21.15.146:0/1183285022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f110c0af5a0 0x7f110c14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 >> 172.21.15.146:0/1183285022 conn(0x7f110c066e00 msgr2=0x7f110c0ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 shutdown_connections 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.469+0000 7f1110d07700 1 -- 172.21.15.146:0/1183285022 wait complete. 2024-08-15T16:59:52.484 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T16:59:52.495 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-15T16:59:51.646407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2385912,"kb_used_data":2884,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752588808,"statfs":{"total":773094113280,"available":770650939392,"internally_reserved":0,"allocated":2953216,"data_stored":1774763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.264320"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413334+0000","last_change":"2024-08-15T16:59:42.409245+0000","last_active":"2024-08-15T16:59:42.413334+0000","last_peered":"2024-08-15T16:59:42.413334+0000","last_clean":"2024-08-15T16:59:42.413334+0000","last_became_active":"2024-08-15T16:59:42.408987+0000","last_became_peered":"2024-08-15T16:59:42.408987+0000","last_unstale":"2024-08-15T16:59:42.413334+0000","last_undegraded":"2024-08-15T16:59:42.413334+0000","last_fullsized":"2024-08-15T16:59:42.413334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:01:07.089558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422299+0000","last_change":"2024-08-15T16:59:42.418957+0000","last_active":"2024-08-15T16:59:42.422299+0000","last_peered":"2024-08-15T16:59:42.422299+0000","last_clean":"2024-08-15T16:59:42.422299+0000","last_became_active":"2024-08-15T16:59:42.418575+0000","last_became_peered":"2024-08-15T16:59:42.418575+0000","last_unstale":"2024-08-15T16:59:42.422299+0000","last_undegraded":"2024-08-15T16:59:42.422299+0000","last_fullsized":"2024-08-15T16:59:42.422299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419909+0000","last_change":"2024-08-15T16:59:42.410104+0000","last_active":"2024-08-15T16:59:42.419909+0000","last_peered":"2024-08-15T16:59:42.419909+0000","last_clean":"2024-08-15T16:59:42.419909+0000","last_became_active":"2024-08-15T16:59:42.409813+0000","last_became_peered":"2024-08-15T16:59:42.409813+0000","last_unstale":"2024-08-15T16:59:42.419909+0000","last_undegraded":"2024-08-15T16:59:42.419909+0000","last_fullsized":"2024-08-15T16:59:42.419909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:00:00.063666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416866+0000","last_change":"2024-08-15T16:59:42.415914+0000","last_active":"2024-08-15T16:59:42.416866+0000","last_peered":"2024-08-15T16:59:42.416866+0000","last_clean":"2024-08-15T16:59:42.416866+0000","last_became_active":"2024-08-15T16:59:42.415646+0000","last_became_peered":"2024-08-15T16:59:42.415646+0000","last_unstale":"2024-08-15T16:59:42.416866+0000","last_undegraded":"2024-08-15T16:59:42.416866+0000","last_fullsized":"2024-08-15T16:59:42.416866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:10:45.984041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413099+0000","last_change":"2024-08-15T16:59:42.406837+0000","last_active":"2024-08-15T16:59:42.413099+0000","last_peered":"2024-08-15T16:59:42.413099+0000","last_clean":"2024-08-15T16:59:42.413099+0000","last_became_active":"2024-08-15T16:59:42.406562+0000","last_became_peered":"2024-08-15T16:59:42.406562+0000","last_unstale":"2024-08-15T16:59:42.413099+0000","last_undegraded":"2024-08-15T16:59:42.413099+0000","last_fullsized":"2024-08-15T16:59:42.413099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:07:47.921825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414600+0000","last_change":"2024-08-15T16:59:42.403625+0000","last_active":"2024-08-15T16:59:42.414600+0000","last_peered":"2024-08-15T16:59:42.414600+0000","last_clean":"2024-08-15T16:59:42.414600+0000","last_became_active":"2024-08-15T16:59:42.403339+0000","last_became_peered":"2024-08-15T16:59:42.403339+0000","last_unstale":"2024-08-15T16:59:42.414600+0000","last_undegraded":"2024-08-15T16:59:42.414600+0000","last_fullsized":"2024-08-15T16:59:42.414600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:30.046447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417227+0000","last_change":"2024-08-15T16:59:42.413170+0000","last_active":"2024-08-15T16:59:42.417227+0000","last_peered":"2024-08-15T16:59:42.417227+0000","last_clean":"2024-08-15T16:59:42.417227+0000","last_became_active":"2024-08-15T16:59:42.409523+0000","last_became_peered":"2024-08-15T16:59:42.409523+0000","last_unstale":"2024-08-15T16:59:42.417227+0000","last_undegraded":"2024-08-15T16:59:42.417227+0000","last_fullsized":"2024-08-15T16:59:42.417227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:43:25.911685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418927+0000","last_change":"2024-08-15T16:59:42.412418+0000","last_active":"2024-08-15T16:59:42.418927+0000","last_peered":"2024-08-15T16:59:42.418927+0000","last_clean":"2024-08-15T16:59:42.418927+0000","last_became_active":"2024-08-15T16:59:42.412174+0000","last_became_peered":"2024-08-15T16:59:42.412174+0000","last_unstale":"2024-08-15T16:59:42.418927+0000","last_undegraded":"2024-08-15T16:59:42.418927+0000","last_fullsized":"2024-08-15T16:59:42.418927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:00:19.102852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419430+0000","last_change":"2024-08-15T16:59:42.416969+0000","last_active":"2024-08-15T16:59:42.419430+0000","last_peered":"2024-08-15T16:59:42.419430+0000","last_clean":"2024-08-15T16:59:42.419430+0000","last_became_active":"2024-08-15T16:59:42.416665+0000","last_became_peered":"2024-08-15T16:59:42.416665+0000","last_unstale":"2024-08-15T16:59:42.419430+0000","last_undegraded":"2024-08-15T16:59:42.419430+0000","last_fullsized":"2024-08-15T16:59:42.419430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:31:42.855850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401340+0000","last_change":"2024-08-15T16:59:42.399470+0000","last_active":"2024-08-15T16:59:42.401340+0000","last_peered":"2024-08-15T16:59:42.401340+0000","last_clean":"2024-08-15T16:59:42.401340+0000","last_became_active":"2024-08-15T16:59:42.398232+0000","last_became_peered":"2024-08-15T16:59:42.398232+0000","last_unstale":"2024-08-15T16:59:42.401340+0000","last_undegraded":"2024-08-15T16:59:42.401340+0000","last_fullsized":"2024-08-15T16:59:42.401340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401337+0000","last_change":"2024-08-15T16:59:42.399759+0000","last_active":"2024-08-15T16:59:42.401337+0000","last_peered":"2024-08-15T16:59:42.401337+0000","last_clean":"2024-08-15T16:59:42.401337+0000","last_became_active":"2024-08-15T16:59:42.398288+0000","last_became_peered":"2024-08-15T16:59:42.398288+0000","last_unstale":"2024-08-15T16:59:42.401337+0000","last_undegraded":"2024-08-15T16:59:42.401337+0000","last_fullsized":"2024-08-15T16:59:42.401337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.402316+0000","last_change":"2024-08-15T16:59:42.397942+0000","last_active":"2024-08-15T16:59:42.402316+0000","last_peered":"2024-08-15T16:59:42.402316+0000","last_clean":"2024-08-15T16:59:42.402316+0000","last_became_active":"2024-08-15T16:59:42.397617+0000","last_became_peered":"2024-08-15T16:59:42.397617+0000","last_unstale":"2024-08-15T16:59:42.402316+0000","last_undegraded":"2024-08-15T16:59:42.402316+0000","last_fullsized":"2024-08-15T16:59:42.402316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:48:39.070398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416917+0000","last_change":"2024-08-15T16:59:42.408020+0000","last_active":"2024-08-15T16:59:42.416917+0000","last_peered":"2024-08-15T16:59:42.416917+0000","last_clean":"2024-08-15T16:59:42.416917+0000","last_became_active":"2024-08-15T16:59:42.407730+0000","last_became_peered":"2024-08-15T16:59:42.407730+0000","last_unstale":"2024-08-15T16:59:42.416917+0000","last_undegraded":"2024-08-15T16:59:42.416917+0000","last_fullsized":"2024-08-15T16:59:42.416917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420317+0000","last_change":"2024-08-15T16:59:42.415216+0000","last_active":"2024-08-15T16:59:42.420317+0000","last_peered":"2024-08-15T16:59:42.420317+0000","last_clean":"2024-08-15T16:59:42.420317+0000","last_became_active":"2024-08-15T16:59:42.414950+0000","last_became_peered":"2024-08-15T16:59:42.414950+0000","last_unstale":"2024-08-15T16:59:42.420317+0000","last_undegraded":"2024-08-15T16:59:42.420317+0000","last_fullsized":"2024-08-15T16:59:42.420317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:47:13.915180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411187+0000","last_change":"2024-08-15T16:59:42.403579+0000","last_active":"2024-08-15T16:59:42.411187+0000","last_peered":"2024-08-15T16:59:42.411187+0000","last_clean":"2024-08-15T16:59:42.411187+0000","last_became_active":"2024-08-15T16:59:42.403244+0000","last_became_peered":"2024-08-15T16:59:42.403244+0000","last_unstale":"2024-08-15T16:59:42.411187+0000","last_undegraded":"2024-08-15T16:59:42.411187+0000","last_fullsized":"2024-08-15T16:59:42.411187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414413+0000","last_change":"2024-08-15T16:59:42.407583+0000","last_active":"2024-08-15T16:59:42.414413+0000","last_peered":"2024-08-15T16:59:42.414413+0000","last_clean":"2024-08-15T16:59:42.414413+0000","last_became_active":"2024-08-15T16:59:42.406689+0000","last_became_peered":"2024-08-15T16:59:42.406689+0000","last_unstale":"2024-08-15T16:59:42.414413+0000","last_undegraded":"2024-08-15T16:59:42.414413+0000","last_fullsized":"2024-08-15T16:59:42.414413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:49:48.769242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414426+0000","last_change":"2024-08-15T16:59:42.404508+0000","last_active":"2024-08-15T16:59:42.414426+0000","last_peered":"2024-08-15T16:59:42.414426+0000","last_clean":"2024-08-15T16:59:42.414426+0000","last_became_active":"2024-08-15T16:59:42.404223+0000","last_became_peered":"2024-08-15T16:59:42.404223+0000","last_unstale":"2024-08-15T16:59:42.414426+0000","last_undegraded":"2024-08-15T16:59:42.414426+0000","last_fullsized":"2024-08-15T16:59:42.414426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:07:35.144380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.403776+0000","last_change":"2024-08-15T16:59:42.400163+0000","last_active":"2024-08-15T16:59:42.403776+0000","last_peered":"2024-08-15T16:59:42.403776+0000","last_clean":"2024-08-15T16:59:42.403776+0000","last_became_active":"2024-08-15T16:59:42.399738+0000","last_became_peered":"2024-08-15T16:59:42.399738+0000","last_unstale":"2024-08-15T16:59:42.403776+0000","last_undegraded":"2024-08-15T16:59:42.403776+0000","last_fullsized":"2024-08-15T16:59:42.403776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:40.046084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413534+0000","last_change":"2024-08-15T16:59:42.403318+0000","last_active":"2024-08-15T16:59:42.413534+0000","last_peered":"2024-08-15T16:59:42.413534+0000","last_clean":"2024-08-15T16:59:42.413534+0000","last_became_active":"2024-08-15T16:59:42.403051+0000","last_became_peered":"2024-08-15T16:59:42.403051+0000","last_unstale":"2024-08-15T16:59:42.413534+0000","last_undegraded":"2024-08-15T16:59:42.413534+0000","last_fullsized":"2024-08-15T16:59:42.413534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401457+0000","last_change":"2024-08-15T16:59:42.397107+0000","last_active":"2024-08-15T16:59:42.401457+0000","last_peered":"2024-08-15T16:59:42.401457+0000","last_clean":"2024-08-15T16:59:42.401457+0000","last_became_active":"2024-08-15T16:59:42.396786+0000","last_became_peered":"2024-08-15T16:59:42.396786+0000","last_unstale":"2024-08-15T16:59:42.401457+0000","last_undegraded":"2024-08-15T16:59:42.401457+0000","last_fullsized":"2024-08-15T16:59:42.401457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422555+0000","last_change":"2024-08-15T16:59:42.418246+0000","last_active":"2024-08-15T16:59:42.422555+0000","last_peered":"2024-08-15T16:59:42.422555+0000","last_clean":"2024-08-15T16:59:42.422555+0000","last_became_active":"2024-08-15T16:59:42.417975+0000","last_became_peered":"2024-08-15T16:59:42.417975+0000","last_unstale":"2024-08-15T16:59:42.422555+0000","last_undegraded":"2024-08-15T16:59:42.422555+0000","last_fullsized":"2024-08-15T16:59:42.422555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:46:29.993181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414618+0000","last_change":"2024-08-15T16:59:42.405054+0000","last_active":"2024-08-15T16:59:42.414618+0000","last_peered":"2024-08-15T16:59:42.414618+0000","last_clean":"2024-08-15T16:59:42.414618+0000","last_became_active":"2024-08-15T16:59:42.404788+0000","last_became_peered":"2024-08-15T16:59:42.404788+0000","last_unstale":"2024-08-15T16:59:42.414618+0000","last_undegraded":"2024-08-15T16:59:42.414618+0000","last_fullsized":"2024-08-15T16:59:42.414618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:49:25.110270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417014+0000","last_change":"2024-08-15T16:59:42.408237+0000","last_active":"2024-08-15T16:59:42.417014+0000","last_peered":"2024-08-15T16:59:42.417014+0000","last_clean":"2024-08-15T16:59:42.417014+0000","last_became_active":"2024-08-15T16:59:42.407321+0000","last_became_peered":"2024-08-15T16:59:42.407321+0000","last_unstale":"2024-08-15T16:59:42.417014+0000","last_undegraded":"2024-08-15T16:59:42.417014+0000","last_fullsized":"2024-08-15T16:59:42.417014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414394+0000","last_change":"2024-08-15T16:59:42.406550+0000","last_active":"2024-08-15T16:59:42.414394+0000","last_peered":"2024-08-15T16:59:42.414394+0000","last_clean":"2024-08-15T16:59:42.414394+0000","last_became_active":"2024-08-15T16:59:42.406280+0000","last_became_peered":"2024-08-15T16:59:42.406280+0000","last_unstale":"2024-08-15T16:59:42.414394+0000","last_undegraded":"2024-08-15T16:59:42.414394+0000","last_fullsized":"2024-08-15T16:59:42.414394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:47:40.336658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416946+0000","last_change":"2024-08-15T16:59:42.408293+0000","last_active":"2024-08-15T16:59:42.416946+0000","last_peered":"2024-08-15T16:59:42.416946+0000","last_clean":"2024-08-15T16:59:42.416946+0000","last_became_active":"2024-08-15T16:59:42.408005+0000","last_became_peered":"2024-08-15T16:59:42.408005+0000","last_unstale":"2024-08-15T16:59:42.416946+0000","last_undegraded":"2024-08-15T16:59:42.416946+0000","last_fullsized":"2024-08-15T16:59:42.416946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:40:55.706726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.408666+0000","last_change":"2024-08-15T16:59:42.403047+0000","last_active":"2024-08-15T16:59:42.408666+0000","last_peered":"2024-08-15T16:59:42.408666+0000","last_clean":"2024-08-15T16:59:42.408666+0000","last_became_active":"2024-08-15T16:59:42.402801+0000","last_became_peered":"2024-08-15T16:59:42.402801+0000","last_unstale":"2024-08-15T16:59:42.408666+0000","last_undegraded":"2024-08-15T16:59:42.408666+0000","last_fullsized":"2024-08-15T16:59:42.408666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416956+0000","last_change":"2024-08-15T16:59:42.411339+0000","last_active":"2024-08-15T16:59:42.416956+0000","last_peered":"2024-08-15T16:59:42.416956+0000","last_clean":"2024-08-15T16:59:42.416956+0000","last_became_active":"2024-08-15T16:59:42.411055+0000","last_became_peered":"2024-08-15T16:59:42.411055+0000","last_unstale":"2024-08-15T16:59:42.416956+0000","last_undegraded":"2024-08-15T16:59:42.416956+0000","last_fullsized":"2024-08-15T16:59:42.416956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417334+0000","last_change":"2024-08-15T16:59:42.411492+0000","last_active":"2024-08-15T16:59:42.417334+0000","last_peered":"2024-08-15T16:59:42.417334+0000","last_clean":"2024-08-15T16:59:42.417334+0000","last_became_active":"2024-08-15T16:59:42.411182+0000","last_became_peered":"2024-08-15T16:59:42.411182+0000","last_unstale":"2024-08-15T16:59:42.417334+0000","last_undegraded":"2024-08-15T16:59:42.417334+0000","last_fullsized":"2024-08-15T16:59:42.417334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420116+0000","last_change":"2024-08-15T16:59:42.412347+0000","last_active":"2024-08-15T16:59:42.420116+0000","last_peered":"2024-08-15T16:59:42.420116+0000","last_clean":"2024-08-15T16:59:42.420116+0000","last_became_active":"2024-08-15T16:59:42.411964+0000","last_became_peered":"2024-08-15T16:59:42.411964+0000","last_unstale":"2024-08-15T16:59:42.420116+0000","last_undegraded":"2024-08-15T16:59:42.420116+0000","last_fullsized":"2024-08-15T16:59:42.420116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:06:21.481110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414638+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.414638+0000","last_peered":"2024-08-15T16:59:42.414638+0000","last_clean":"2024-08-15T16:59:42.414638+0000","last_became_active":"2024-08-15T16:59:42.407063+0000","last_became_peered":"2024-08-15T16:59:42.407063+0000","last_unstale":"2024-08-15T16:59:42.414638+0000","last_undegraded":"2024-08-15T16:59:42.414638+0000","last_fullsized":"2024-08-15T16:59:42.414638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:54:52.191383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.405736+0000","last_change":"2024-08-15T16:59:42.398430+0000","last_active":"2024-08-15T16:59:42.405736+0000","last_peered":"2024-08-15T16:59:42.405736+0000","last_clean":"2024-08-15T16:59:42.405736+0000","last_became_active":"2024-08-15T16:59:42.397995+0000","last_became_peered":"2024-08-15T16:59:42.397995+0000","last_unstale":"2024-08-15T16:59:42.405736+0000","last_undegraded":"2024-08-15T16:59:42.405736+0000","last_fullsized":"2024-08-15T16:59:42.405736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:02.439971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418173+0000","last_change":"2024-08-15T16:59:42.417359+0000","last_active":"2024-08-15T16:59:42.418173+0000","last_peered":"2024-08-15T16:59:42.418173+0000","last_clean":"2024-08-15T16:59:42.418173+0000","last_became_active":"2024-08-15T16:59:42.416784+0000","last_became_peered":"2024-08-15T16:59:42.416784+0000","last_unstale":"2024-08-15T16:59:42.418173+0000","last_undegraded":"2024-08-15T16:59:42.418173+0000","last_fullsized":"2024-08-15T16:59:42.418173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400566+0000","last_change":"2024-08-15T16:59:42.396288+0000","last_active":"2024-08-15T16:59:42.400566+0000","last_peered":"2024-08-15T16:59:42.400566+0000","last_clean":"2024-08-15T16:59:42.400566+0000","last_became_active":"2024-08-15T16:59:42.395944+0000","last_became_peered":"2024-08-15T16:59:42.395944+0000","last_unstale":"2024-08-15T16:59:42.400566+0000","last_undegraded":"2024-08-15T16:59:42.400566+0000","last_fullsized":"2024-08-15T16:59:42.400566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419597+0000","last_change":"2024-08-15T16:59:41.381424+0000","last_active":"2024-08-15T16:59:42.419597+0000","last_peered":"2024-08-15T16:59:42.419597+0000","last_clean":"2024-08-15T16:59:42.419597+0000","last_became_active":"2024-08-15T16:59:41.381154+0000","last_became_peered":"2024-08-15T16:59:41.381154+0000","last_unstale":"2024-08-15T16:59:42.419597+0000","last_undegraded":"2024-08-15T16:59:42.419597+0000","last_fullsized":"2024-08-15T16:59:42.419597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:11:23.395735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420098+0000","last_change":"2024-08-15T16:59:42.412224+0000","last_active":"2024-08-15T16:59:42.420098+0000","last_peered":"2024-08-15T16:59:42.420098+0000","last_clean":"2024-08-15T16:59:42.420098+0000","last_became_active":"2024-08-15T16:59:42.411384+0000","last_became_peered":"2024-08-15T16:59:42.411384+0000","last_unstale":"2024-08-15T16:59:42.420098+0000","last_undegraded":"2024-08-15T16:59:42.420098+0000","last_fullsized":"2024-08-15T16:59:42.420098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:28:38.772967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.415389+0000","last_change":"2024-08-15T16:59:41.370657+0000","last_active":"2024-08-15T16:59:42.415389+0000","last_peered":"2024-08-15T16:59:42.415389+0000","last_clean":"2024-08-15T16:59:42.415389+0000","last_became_active":"2024-08-15T16:59:41.370303+0000","last_became_peered":"2024-08-15T16:59:41.370303+0000","last_unstale":"2024-08-15T16:59:42.415389+0000","last_undegraded":"2024-08-15T16:59:42.415389+0000","last_fullsized":"2024-08-15T16:59:42.415389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398754+0000","last_change":"2024-08-15T16:59:41.376440+0000","last_active":"2024-08-15T16:59:42.398754+0000","last_peered":"2024-08-15T16:59:42.398754+0000","last_clean":"2024-08-15T16:59:42.398754+0000","last_became_active":"2024-08-15T16:59:41.375356+0000","last_became_peered":"2024-08-15T16:59:41.375356+0000","last_unstale":"2024-08-15T16:59:42.398754+0000","last_undegraded":"2024-08-15T16:59:42.398754+0000","last_fullsized":"2024-08-15T16:59:42.398754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418845+0000","last_change":"2024-08-15T16:59:42.409055+0000","last_active":"2024-08-15T16:59:42.418845+0000","last_peered":"2024-08-15T16:59:42.418845+0000","last_clean":"2024-08-15T16:59:42.418845+0000","last_became_active":"2024-08-15T16:59:42.408704+0000","last_became_peered":"2024-08-15T16:59:42.408704+0000","last_unstale":"2024-08-15T16:59:42.418845+0000","last_undegraded":"2024-08-15T16:59:42.418845+0000","last_fullsized":"2024-08-15T16:59:42.418845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.388705+0000","last_change":"2024-08-15T16:59:41.372714+0000","last_active":"2024-08-15T16:59:42.388705+0000","last_peered":"2024-08-15T16:59:42.388705+0000","last_clean":"2024-08-15T16:59:42.388705+0000","last_became_active":"2024-08-15T16:59:41.372369+0000","last_became_peered":"2024-08-15T16:59:41.372369+0000","last_unstale":"2024-08-15T16:59:42.388705+0000","last_undegraded":"2024-08-15T16:59:42.388705+0000","last_fullsized":"2024-08-15T16:59:42.388705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:30:18.195005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417279+0000","last_change":"2024-08-15T16:59:42.412973+0000","last_active":"2024-08-15T16:59:42.417279+0000","last_peered":"2024-08-15T16:59:42.417279+0000","last_clean":"2024-08-15T16:59:42.417279+0000","last_became_active":"2024-08-15T16:59:42.412668+0000","last_became_peered":"2024-08-15T16:59:42.412668+0000","last_unstale":"2024-08-15T16:59:42.417279+0000","last_undegraded":"2024-08-15T16:59:42.417279+0000","last_fullsized":"2024-08-15T16:59:42.417279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.394506+0000","last_change":"2024-08-15T16:59:41.371198+0000","last_active":"2024-08-15T16:59:42.394506+0000","last_peered":"2024-08-15T16:59:42.394506+0000","last_clean":"2024-08-15T16:59:42.394506+0000","last_became_active":"2024-08-15T16:59:41.370260+0000","last_became_peered":"2024-08-15T16:59:41.370260+0000","last_unstale":"2024-08-15T16:59:42.394506+0000","last_undegraded":"2024-08-15T16:59:42.394506+0000","last_fullsized":"2024-08-15T16:59:42.394506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414362+0000","last_change":"2024-08-15T16:59:42.403938+0000","last_active":"2024-08-15T16:59:42.414362+0000","last_peered":"2024-08-15T16:59:42.414362+0000","last_clean":"2024-08-15T16:59:42.414362+0000","last_became_active":"2024-08-15T16:59:42.403693+0000","last_became_peered":"2024-08-15T16:59:42.403693+0000","last_unstale":"2024-08-15T16:59:42.414362+0000","last_undegraded":"2024-08-15T16:59:42.414362+0000","last_fullsized":"2024-08-15T16:59:42.414362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:09:46.202413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.390318+0000","last_change":"2024-08-15T16:59:41.370700+0000","last_active":"2024-08-15T16:59:42.390318+0000","last_peered":"2024-08-15T16:59:42.390318+0000","last_clean":"2024-08-15T16:59:42.390318+0000","last_became_active":"2024-08-15T16:59:41.370333+0000","last_became_peered":"2024-08-15T16:59:41.370333+0000","last_unstale":"2024-08-15T16:59:42.390318+0000","last_undegraded":"2024-08-15T16:59:42.390318+0000","last_fullsized":"2024-08-15T16:59:42.390318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:25:40.170510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420410+0000","last_change":"2024-08-15T16:59:42.415577+0000","last_active":"2024-08-15T16:59:42.420410+0000","last_peered":"2024-08-15T16:59:42.420410+0000","last_clean":"2024-08-15T16:59:42.420410+0000","last_became_active":"2024-08-15T16:59:42.415259+0000","last_became_peered":"2024-08-15T16:59:42.415259+0000","last_unstale":"2024-08-15T16:59:42.420410+0000","last_undegraded":"2024-08-15T16:59:42.420410+0000","last_fullsized":"2024-08-15T16:59:42.420410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:53:31.974939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420880+0000","last_change":"2024-08-15T16:59:41.382088+0000","last_active":"2024-08-15T16:59:42.420880+0000","last_peered":"2024-08-15T16:59:42.420880+0000","last_clean":"2024-08-15T16:59:42.420880+0000","last_became_active":"2024-08-15T16:59:41.380881+0000","last_became_peered":"2024-08-15T16:59:41.380881+0000","last_unstale":"2024-08-15T16:59:42.420880+0000","last_undegraded":"2024-08-15T16:59:42.420880+0000","last_fullsized":"2024-08-15T16:59:42.420880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.404994+0000","last_change":"2024-08-15T16:59:42.403887+0000","last_active":"2024-08-15T16:59:42.404994+0000","last_peered":"2024-08-15T16:59:42.404994+0000","last_clean":"2024-08-15T16:59:42.404994+0000","last_became_active":"2024-08-15T16:59:42.403619+0000","last_became_peered":"2024-08-15T16:59:42.403619+0000","last_unstale":"2024-08-15T16:59:42.404994+0000","last_undegraded":"2024-08-15T16:59:42.404994+0000","last_fullsized":"2024-08-15T16:59:42.404994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421139+0000","last_change":"2024-08-15T16:59:41.379757+0000","last_active":"2024-08-15T16:59:42.421139+0000","last_peered":"2024-08-15T16:59:42.421139+0000","last_clean":"2024-08-15T16:59:42.421139+0000","last_became_active":"2024-08-15T16:59:41.379393+0000","last_became_peered":"2024-08-15T16:59:41.379393+0000","last_unstale":"2024-08-15T16:59:42.421139+0000","last_undegraded":"2024-08-15T16:59:42.421139+0000","last_fullsized":"2024-08-15T16:59:42.421139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418889+0000","last_change":"2024-08-15T16:59:42.413445+0000","last_active":"2024-08-15T16:59:42.418889+0000","last_peered":"2024-08-15T16:59:42.418889+0000","last_clean":"2024-08-15T16:59:42.418889+0000","last_became_active":"2024-08-15T16:59:42.413176+0000","last_became_peered":"2024-08-15T16:59:42.413176+0000","last_unstale":"2024-08-15T16:59:42.418889+0000","last_undegraded":"2024-08-15T16:59:42.418889+0000","last_fullsized":"2024-08-15T16:59:42.418889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:01:08.921681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420263+0000","last_change":"2024-08-15T16:59:41.381453+0000","last_active":"2024-08-15T16:59:42.420263+0000","last_peered":"2024-08-15T16:59:42.420263+0000","last_clean":"2024-08-15T16:59:42.420263+0000","last_became_active":"2024-08-15T16:59:41.381090+0000","last_became_peered":"2024-08-15T16:59:41.381090+0000","last_unstale":"2024-08-15T16:59:42.420263+0000","last_undegraded":"2024-08-15T16:59:42.420263+0000","last_fullsized":"2024-08-15T16:59:42.420263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:05:07.421014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420060+0000","last_change":"2024-08-15T16:59:42.411131+0000","last_active":"2024-08-15T16:59:42.420060+0000","last_peered":"2024-08-15T16:59:42.420060+0000","last_clean":"2024-08-15T16:59:42.420060+0000","last_became_active":"2024-08-15T16:59:42.410756+0000","last_became_peered":"2024-08-15T16:59:42.410756+0000","last_unstale":"2024-08-15T16:59:42.420060+0000","last_undegraded":"2024-08-15T16:59:42.420060+0000","last_fullsized":"2024-08-15T16:59:42.420060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:19:24.263405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391610+0000","last_change":"2024-08-15T16:59:41.373319+0000","last_active":"2024-08-15T16:59:42.391610+0000","last_peered":"2024-08-15T16:59:42.391610+0000","last_clean":"2024-08-15T16:59:42.391610+0000","last_became_active":"2024-08-15T16:59:41.369550+0000","last_became_peered":"2024-08-15T16:59:41.369550+0000","last_unstale":"2024-08-15T16:59:42.391610+0000","last_undegraded":"2024-08-15T16:59:42.391610+0000","last_fullsized":"2024-08-15T16:59:42.391610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:16:34.424937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413118+0000","last_change":"2024-08-15T16:59:42.407307+0000","last_active":"2024-08-15T16:59:42.413118+0000","last_peered":"2024-08-15T16:59:42.413118+0000","last_clean":"2024-08-15T16:59:42.413118+0000","last_became_active":"2024-08-15T16:59:42.407033+0000","last_became_peered":"2024-08-15T16:59:42.407033+0000","last_unstale":"2024-08-15T16:59:42.413118+0000","last_undegraded":"2024-08-15T16:59:42.413118+0000","last_fullsized":"2024-08-15T16:59:42.413118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:47:58.883271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.392955+0000","last_change":"2024-08-15T16:59:41.370530+0000","last_active":"2024-08-15T16:59:42.392955+0000","last_peered":"2024-08-15T16:59:42.392955+0000","last_clean":"2024-08-15T16:59:42.392955+0000","last_became_active":"2024-08-15T16:59:41.369807+0000","last_became_peered":"2024-08-15T16:59:41.369807+0000","last_unstale":"2024-08-15T16:59:42.392955+0000","last_undegraded":"2024-08-15T16:59:42.392955+0000","last_fullsized":"2024-08-15T16:59:42.392955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418194+0000","last_change":"2024-08-15T16:59:42.417158+0000","last_active":"2024-08-15T16:59:42.418194+0000","last_peered":"2024-08-15T16:59:42.418194+0000","last_clean":"2024-08-15T16:59:42.418194+0000","last_became_active":"2024-08-15T16:59:42.416838+0000","last_became_peered":"2024-08-15T16:59:42.416838+0000","last_unstale":"2024-08-15T16:59:42.418194+0000","last_undegraded":"2024-08-15T16:59:42.418194+0000","last_fullsized":"2024-08-15T16:59:42.418194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422895+0000","last_change":"2024-08-15T16:59:41.377962+0000","last_active":"2024-08-15T16:59:42.422895+0000","last_peered":"2024-08-15T16:59:42.422895+0000","last_clean":"2024-08-15T16:59:42.422895+0000","last_became_active":"2024-08-15T16:59:41.377699+0000","last_became_peered":"2024-08-15T16:59:41.377699+0000","last_unstale":"2024-08-15T16:59:42.422895+0000","last_undegraded":"2024-08-15T16:59:42.422895+0000","last_fullsized":"2024-08-15T16:59:42.422895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413284+0000","last_change":"2024-08-15T16:59:42.407316+0000","last_active":"2024-08-15T16:59:42.413284+0000","last_peered":"2024-08-15T16:59:42.413284+0000","last_clean":"2024-08-15T16:59:42.413284+0000","last_became_active":"2024-08-15T16:59:42.407049+0000","last_became_peered":"2024-08-15T16:59:42.407049+0000","last_unstale":"2024-08-15T16:59:42.413284+0000","last_undegraded":"2024-08-15T16:59:42.413284+0000","last_fullsized":"2024-08-15T16:59:42.413284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:48.938421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418920+0000","last_change":"2024-08-15T16:59:41.373342+0000","last_active":"2024-08-15T16:59:42.418920+0000","last_peered":"2024-08-15T16:59:42.418920+0000","last_clean":"2024-08-15T16:59:42.418920+0000","last_became_active":"2024-08-15T16:59:41.373010+0000","last_became_peered":"2024-08-15T16:59:41.373010+0000","last_unstale":"2024-08-15T16:59:42.418920+0000","last_undegraded":"2024-08-15T16:59:42.418920+0000","last_fullsized":"2024-08-15T16:59:42.418920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:38:43.125122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413438+0000","last_change":"2024-08-15T16:59:42.404605+0000","last_active":"2024-08-15T16:59:42.413438+0000","last_peered":"2024-08-15T16:59:42.413438+0000","last_clean":"2024-08-15T16:59:42.413438+0000","last_became_active":"2024-08-15T16:59:42.404054+0000","last_became_peered":"2024-08-15T16:59:42.404054+0000","last_unstale":"2024-08-15T16:59:42.413438+0000","last_undegraded":"2024-08-15T16:59:42.413438+0000","last_fullsized":"2024-08-15T16:59:42.413438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.597396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419684+0000","last_change":"2024-08-15T16:59:41.381959+0000","last_active":"2024-08-15T16:59:42.419684+0000","last_peered":"2024-08-15T16:59:42.419684+0000","last_clean":"2024-08-15T16:59:42.419684+0000","last_became_active":"2024-08-15T16:59:41.381634+0000","last_became_peered":"2024-08-15T16:59:41.381634+0000","last_unstale":"2024-08-15T16:59:42.419684+0000","last_undegraded":"2024-08-15T16:59:42.419684+0000","last_fullsized":"2024-08-15T16:59:42.419684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.413316+0000","last_change":"2024-08-15T16:59:42.408082+0000","last_active":"2024-08-15T16:59:42.413316+0000","last_peered":"2024-08-15T16:59:42.413316+0000","last_clean":"2024-08-15T16:59:42.413316+0000","last_became_active":"2024-08-15T16:59:42.407840+0000","last_became_peered":"2024-08-15T16:59:42.407840+0000","last_unstale":"2024-08-15T16:59:42.413316+0000","last_undegraded":"2024-08-15T16:59:42.413316+0000","last_fullsized":"2024-08-15T16:59:42.413316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:48:32.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422292+0000","last_change":"2024-08-15T16:59:41.381232+0000","last_active":"2024-08-15T16:59:42.422292+0000","last_peered":"2024-08-15T16:59:42.422292+0000","last_clean":"2024-08-15T16:59:42.422292+0000","last_became_active":"2024-08-15T16:59:41.380497+0000","last_became_peered":"2024-08-15T16:59:41.380497+0000","last_unstale":"2024-08-15T16:59:42.422292+0000","last_undegraded":"2024-08-15T16:59:42.422292+0000","last_fullsized":"2024-08-15T16:59:42.422292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414529+0000","last_change":"2024-08-15T16:59:42.410719+0000","last_active":"2024-08-15T16:59:42.414529+0000","last_peered":"2024-08-15T16:59:42.414529+0000","last_clean":"2024-08-15T16:59:42.414529+0000","last_became_active":"2024-08-15T16:59:42.410323+0000","last_became_peered":"2024-08-15T16:59:42.410323+0000","last_unstale":"2024-08-15T16:59:42.414529+0000","last_undegraded":"2024-08-15T16:59:42.414529+0000","last_fullsized":"2024-08-15T16:59:42.414529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:22:43.534462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387934+0000","last_change":"2024-08-15T16:59:41.375212+0000","last_active":"2024-08-15T16:59:42.387934+0000","last_peered":"2024-08-15T16:59:42.387934+0000","last_clean":"2024-08-15T16:59:42.387934+0000","last_became_active":"2024-08-15T16:59:41.374811+0000","last_became_peered":"2024-08-15T16:59:41.374811+0000","last_unstale":"2024-08-15T16:59:42.387934+0000","last_undegraded":"2024-08-15T16:59:42.387934+0000","last_fullsized":"2024-08-15T16:59:42.387934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:12:48.546187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.419483+0000","last_change":"2024-08-15T16:59:42.416559+0000","last_active":"2024-08-15T16:59:42.419483+0000","last_peered":"2024-08-15T16:59:42.419483+0000","last_clean":"2024-08-15T16:59:42.419483+0000","last_became_active":"2024-08-15T16:59:42.416241+0000","last_became_peered":"2024-08-15T16:59:42.416241+0000","last_unstale":"2024-08-15T16:59:42.419483+0000","last_undegraded":"2024-08-15T16:59:42.419483+0000","last_fullsized":"2024-08-15T16:59:42.419483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:30:00.824354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395661+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T16:59:42.395661+0000","last_peered":"2024-08-15T16:59:42.395661+0000","last_clean":"2024-08-15T16:59:42.395661+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T16:59:42.395661+0000","last_undegraded":"2024-08-15T16:59:42.395661+0000","last_fullsized":"2024-08-15T16:59:42.395661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.386944+0000","last_change":"2024-08-15T16:59:41.374273+0000","last_active":"2024-08-15T16:59:42.386944+0000","last_peered":"2024-08-15T16:59:42.386944+0000","last_clean":"2024-08-15T16:59:42.386944+0000","last_became_active":"2024-08-15T16:59:41.373947+0000","last_became_peered":"2024-08-15T16:59:41.373947+0000","last_unstale":"2024-08-15T16:59:42.386944+0000","last_undegraded":"2024-08-15T16:59:42.386944+0000","last_fullsized":"2024-08-15T16:59:42.386944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:59:06.484884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417638+0000","last_change":"2024-08-15T16:59:42.412036+0000","last_active":"2024-08-15T16:59:42.417638+0000","last_peered":"2024-08-15T16:59:42.417638+0000","last_clean":"2024-08-15T16:59:42.417638+0000","last_became_active":"2024-08-15T16:59:42.411769+0000","last_became_peered":"2024-08-15T16:59:42.411769+0000","last_unstale":"2024-08-15T16:59:42.417638+0000","last_undegraded":"2024-08-15T16:59:42.417638+0000","last_fullsized":"2024-08-15T16:59:42.417638+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:15:27.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395564+0000","last_change":"2024-08-15T16:59:41.384227+0000","last_active":"2024-08-15T16:59:42.395564+0000","last_peered":"2024-08-15T16:59:42.395564+0000","last_clean":"2024-08-15T16:59:42.395564+0000","last_became_active":"2024-08-15T16:59:41.384013+0000","last_became_peered":"2024-08-15T16:59:41.384013+0000","last_unstale":"2024-08-15T16:59:42.395564+0000","last_undegraded":"2024-08-15T16:59:42.395564+0000","last_fullsized":"2024-08-15T16:59:42.395564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.417479+0000","last_change":"2024-08-15T16:59:42.406032+0000","last_active":"2024-08-15T16:59:42.417479+0000","last_peered":"2024-08-15T16:59:42.417479+0000","last_clean":"2024-08-15T16:59:42.417479+0000","last_became_active":"2024-08-15T16:59:42.405742+0000","last_became_peered":"2024-08-15T16:59:42.405742+0000","last_unstale":"2024-08-15T16:59:42.417479+0000","last_undegraded":"2024-08-15T16:59:42.417479+0000","last_fullsized":"2024-08-15T16:59:42.417479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:42:07.291054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.387905+0000","last_change":"2024-08-15T16:59:41.382128+0000","last_active":"2024-08-15T16:59:42.387905+0000","last_peered":"2024-08-15T16:59:42.387905+0000","last_clean":"2024-08-15T16:59:42.387905+0000","last_became_active":"2024-08-15T16:59:41.381700+0000","last_became_peered":"2024-08-15T16:59:41.381700+0000","last_unstale":"2024-08-15T16:59:42.387905+0000","last_undegraded":"2024-08-15T16:59:42.387905+0000","last_fullsized":"2024-08-15T16:59:42.387905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414769+0000","last_change":"2024-08-15T16:59:42.407654+0000","last_active":"2024-08-15T16:59:42.414769+0000","last_peered":"2024-08-15T16:59:42.414769+0000","last_clean":"2024-08-15T16:59:42.414769+0000","last_became_active":"2024-08-15T16:59:42.405202+0000","last_became_peered":"2024-08-15T16:59:42.405202+0000","last_unstale":"2024-08-15T16:59:42.414769+0000","last_undegraded":"2024-08-15T16:59:42.414769+0000","last_fullsized":"2024-08-15T16:59:42.414769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.391657+0000","last_change":"2024-08-15T16:59:41.374124+0000","last_active":"2024-08-15T16:59:42.391657+0000","last_peered":"2024-08-15T16:59:42.391657+0000","last_clean":"2024-08-15T16:59:42.391657+0000","last_became_active":"2024-08-15T16:59:41.373620+0000","last_became_peered":"2024-08-15T16:59:41.373620+0000","last_unstale":"2024-08-15T16:59:42.391657+0000","last_undegraded":"2024-08-15T16:59:42.391657+0000","last_fullsized":"2024-08-15T16:59:42.391657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:55:35.560885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.410018+0000","last_change":"2024-08-15T16:59:42.408577+0000","last_active":"2024-08-15T16:59:42.410018+0000","last_peered":"2024-08-15T16:59:42.410018+0000","last_clean":"2024-08-15T16:59:42.410018+0000","last_became_active":"2024-08-15T16:59:42.408272+0000","last_became_peered":"2024-08-15T16:59:42.408272+0000","last_unstale":"2024-08-15T16:59:42.410018+0000","last_undegraded":"2024-08-15T16:59:42.410018+0000","last_fullsized":"2024-08-15T16:59:42.410018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:54:55.794089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.396515+0000","last_change":"2024-08-15T16:59:41.383124+0000","last_active":"2024-08-15T16:59:42.396515+0000","last_peered":"2024-08-15T16:59:42.396515+0000","last_clean":"2024-08-15T16:59:42.396515+0000","last_became_active":"2024-08-15T16:59:41.382696+0000","last_became_peered":"2024-08-15T16:59:41.382696+0000","last_unstale":"2024-08-15T16:59:42.396515+0000","last_undegraded":"2024-08-15T16:59:42.396515+0000","last_fullsized":"2024-08-15T16:59:42.396515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:01.430020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.414332+0000","last_change":"2024-08-15T16:59:42.406595+0000","last_active":"2024-08-15T16:59:42.414332+0000","last_peered":"2024-08-15T16:59:42.414332+0000","last_clean":"2024-08-15T16:59:42.414332+0000","last_became_active":"2024-08-15T16:59:42.406353+0000","last_became_peered":"2024-08-15T16:59:42.406353+0000","last_unstale":"2024-08-15T16:59:42.414332+0000","last_undegraded":"2024-08-15T16:59:42.414332+0000","last_fullsized":"2024-08-15T16:59:42.414332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:25:43.883310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.398385+0000","last_change":"2024-08-15T16:59:41.381997+0000","last_active":"2024-08-15T16:59:42.398385+0000","last_peered":"2024-08-15T16:59:42.398385+0000","last_clean":"2024-08-15T16:59:42.398385+0000","last_became_active":"2024-08-15T16:59:41.381679+0000","last_became_peered":"2024-08-15T16:59:41.381679+0000","last_unstale":"2024-08-15T16:59:42.398385+0000","last_undegraded":"2024-08-15T16:59:42.398385+0000","last_fullsized":"2024-08-15T16:59:42.398385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:14:39.036133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.411157+0000","last_change":"2024-08-15T16:59:42.404468+0000","last_active":"2024-08-15T16:59:42.411157+0000","last_peered":"2024-08-15T16:59:42.411157+0000","last_clean":"2024-08-15T16:59:42.411157+0000","last_became_active":"2024-08-15T16:59:42.403172+0000","last_became_peered":"2024-08-15T16:59:42.403172+0000","last_unstale":"2024-08-15T16:59:42.411157+0000","last_undegraded":"2024-08-15T16:59:42.411157+0000","last_fullsized":"2024-08-15T16:59:42.411157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:25:48.621984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-15T16:59:43.395993+0000","last_change":"2024-08-15T16:59:41.379177+0000","last_active":"2024-08-15T16:59:43.395993+0000","last_peered":"2024-08-15T16:59:43.395993+0000","last_clean":"2024-08-15T16:59:43.395993+0000","last_became_active":"2024-08-15T16:59:41.378909+0000","last_became_peered":"2024-08-15T16:59:41.378909+0000","last_unstale":"2024-08-15T16:59:43.395993+0000","last_undegraded":"2024-08-15T16:59:43.395993+0000","last_fullsized":"2024-08-15T16:59:43.395993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416728+0000","last_change":"2024-08-15T16:59:42.407782+0000","last_active":"2024-08-15T16:59:42.416728+0000","last_peered":"2024-08-15T16:59:42.416728+0000","last_clean":"2024-08-15T16:59:42.416728+0000","last_became_active":"2024-08-15T16:59:42.407483+0000","last_became_peered":"2024-08-15T16:59:42.407483+0000","last_unstale":"2024-08-15T16:59:42.416728+0000","last_undegraded":"2024-08-15T16:59:42.416728+0000","last_fullsized":"2024-08-15T16:59:42.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:27:32.485648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420095+0000","last_change":"2024-08-15T16:59:41.379440+0000","last_active":"2024-08-15T16:59:42.420095+0000","last_peered":"2024-08-15T16:59:42.420095+0000","last_clean":"2024-08-15T16:59:42.420095+0000","last_became_active":"2024-08-15T16:59:41.379116+0000","last_became_peered":"2024-08-15T16:59:41.379116+0000","last_unstale":"2024-08-15T16:59:42.420095+0000","last_undegraded":"2024-08-15T16:59:42.420095+0000","last_fullsized":"2024-08-15T16:59:42.420095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420125+0000","last_change":"2024-08-15T16:59:42.414998+0000","last_active":"2024-08-15T16:59:42.420125+0000","last_peered":"2024-08-15T16:59:42.420125+0000","last_clean":"2024-08-15T16:59:42.420125+0000","last_became_active":"2024-08-15T16:59:42.414696+0000","last_became_peered":"2024-08-15T16:59:42.414696+0000","last_unstale":"2024-08-15T16:59:42.420125+0000","last_undegraded":"2024-08-15T16:59:42.420125+0000","last_fullsized":"2024-08-15T16:59:42.420125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:42:39.871776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395563+0000","last_change":"2024-08-15T16:59:41.381044+0000","last_active":"2024-08-15T16:59:42.395563+0000","last_peered":"2024-08-15T16:59:42.395563+0000","last_clean":"2024-08-15T16:59:42.395563+0000","last_became_active":"2024-08-15T16:59:41.380776+0000","last_became_peered":"2024-08-15T16:59:41.380776+0000","last_unstale":"2024-08-15T16:59:42.395563+0000","last_undegraded":"2024-08-15T16:59:42.395563+0000","last_fullsized":"2024-08-15T16:59:42.395563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:21:56.894684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.409831+0000","last_change":"2024-08-15T16:59:42.408779+0000","last_active":"2024-08-15T16:59:42.409831+0000","last_peered":"2024-08-15T16:59:42.409831+0000","last_clean":"2024-08-15T16:59:42.409831+0000","last_became_active":"2024-08-15T16:59:42.408345+0000","last_became_peered":"2024-08-15T16:59:42.408345+0000","last_unstale":"2024-08-15T16:59:42.409831+0000","last_undegraded":"2024-08-15T16:59:42.409831+0000","last_fullsized":"2024-08-15T16:59:42.409831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:55:03.519854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395271+0000","last_change":"2024-08-15T16:59:41.382297+0000","last_active":"2024-08-15T16:59:42.395271+0000","last_peered":"2024-08-15T16:59:42.395271+0000","last_clean":"2024-08-15T16:59:42.395271+0000","last_became_active":"2024-08-15T16:59:41.381812+0000","last_became_peered":"2024-08-15T16:59:41.381812+0000","last_unstale":"2024-08-15T16:59:42.395271+0000","last_undegraded":"2024-08-15T16:59:42.395271+0000","last_fullsized":"2024-08-15T16:59:42.395271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:26:07.419415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.418823+0000","last_change":"2024-08-15T16:59:42.402821+0000","last_active":"2024-08-15T16:59:42.418823+0000","last_peered":"2024-08-15T16:59:42.418823+0000","last_clean":"2024-08-15T16:59:42.418823+0000","last_became_active":"2024-08-15T16:59:42.402044+0000","last_became_peered":"2024-08-15T16:59:42.402044+0000","last_unstale":"2024-08-15T16:59:42.418823+0000","last_undegraded":"2024-08-15T16:59:42.418823+0000","last_fullsized":"2024-08-15T16:59:42.418823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.395855+0000","last_change":"2024-08-15T16:59:41.383793+0000","last_active":"2024-08-15T16:59:42.395855+0000","last_peered":"2024-08-15T16:59:42.395855+0000","last_clean":"2024-08-15T16:59:42.395855+0000","last_became_active":"2024-08-15T16:59:41.383431+0000","last_became_peered":"2024-08-15T16:59:41.383431+0000","last_unstale":"2024-08-15T16:59:42.395855+0000","last_undegraded":"2024-08-15T16:59:42.395855+0000","last_fullsized":"2024-08-15T16:59:42.395855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:00:47.060047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422596+0000","last_change":"2024-08-15T16:59:42.419454+0000","last_active":"2024-08-15T16:59:42.422596+0000","last_peered":"2024-08-15T16:59:42.422596+0000","last_clean":"2024-08-15T16:59:42.422596+0000","last_became_active":"2024-08-15T16:59:42.419157+0000","last_became_peered":"2024-08-15T16:59:42.419157+0000","last_unstale":"2024-08-15T16:59:42.422596+0000","last_undegraded":"2024-08-15T16:59:42.422596+0000","last_fullsized":"2024-08-15T16:59:42.422596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:57.069114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421197+0000","last_change":"2024-08-15T16:59:41.382836+0000","last_active":"2024-08-15T16:59:42.421197+0000","last_peered":"2024-08-15T16:59:42.421197+0000","last_clean":"2024-08-15T16:59:42.421197+0000","last_became_active":"2024-08-15T16:59:41.382595+0000","last_became_peered":"2024-08-15T16:59:41.382595+0000","last_unstale":"2024-08-15T16:59:42.421197+0000","last_undegraded":"2024-08-15T16:59:42.421197+0000","last_fullsized":"2024-08-15T16:59:42.421197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:09:40.716933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.401292+0000","last_change":"2024-08-15T16:59:42.399359+0000","last_active":"2024-08-15T16:59:42.401292+0000","last_peered":"2024-08-15T16:59:42.401292+0000","last_clean":"2024-08-15T16:59:42.401292+0000","last_became_active":"2024-08-15T16:59:42.398800+0000","last_became_peered":"2024-08-15T16:59:42.398800+0000","last_unstale":"2024-08-15T16:59:42.401292+0000","last_undegraded":"2024-08-15T16:59:42.401292+0000","last_fullsized":"2024-08-15T16:59:42.401292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:16:53.332892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.397841+0000","last_change":"2024-08-15T16:59:42.396158+0000","last_active":"2024-08-15T16:59:42.397841+0000","last_peered":"2024-08-15T16:59:42.397841+0000","last_clean":"2024-08-15T16:59:42.397841+0000","last_became_active":"2024-08-15T16:59:42.395850+0000","last_became_peered":"2024-08-15T16:59:42.395850+0000","last_unstale":"2024-08-15T16:59:42.397841+0000","last_undegraded":"2024-08-15T16:59:42.397841+0000","last_fullsized":"2024-08-15T16:59:42.397841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:39.678214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.422026+0000","last_change":"2024-08-15T16:59:41.385558+0000","last_active":"2024-08-15T16:59:42.422026+0000","last_peered":"2024-08-15T16:59:42.422026+0000","last_clean":"2024-08-15T16:59:42.422026+0000","last_became_active":"2024-08-15T16:59:41.385303+0000","last_became_peered":"2024-08-15T16:59:41.385303+0000","last_unstale":"2024-08-15T16:59:42.422026+0000","last_undegraded":"2024-08-15T16:59:42.422026+0000","last_fullsized":"2024-08-15T16:59:42.422026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:34:37.827415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.400920+0000","last_change":"2024-08-15T16:59:42.396765+0000","last_active":"2024-08-15T16:59:42.400920+0000","last_peered":"2024-08-15T16:59:42.400920+0000","last_clean":"2024-08-15T16:59:42.400920+0000","last_became_active":"2024-08-15T16:59:42.396133+0000","last_became_peered":"2024-08-15T16:59:42.396133+0000","last_unstale":"2024-08-15T16:59:42.400920+0000","last_undegraded":"2024-08-15T16:59:42.400920+0000","last_fullsized":"2024-08-15T16:59:42.400920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:38:03.347839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420319+0000","last_change":"2024-08-15T16:59:41.381180+0000","last_active":"2024-08-15T16:59:42.420319+0000","last_peered":"2024-08-15T16:59:42.420319+0000","last_clean":"2024-08-15T16:59:42.420319+0000","last_became_active":"2024-08-15T16:59:41.381011+0000","last_became_peered":"2024-08-15T16:59:41.381011+0000","last_unstale":"2024-08-15T16:59:42.420319+0000","last_undegraded":"2024-08-15T16:59:42.420319+0000","last_fullsized":"2024-08-15T16:59:42.420319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:21:56.796383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420390+0000","last_change":"2024-08-15T16:59:42.416366+0000","last_active":"2024-08-15T16:59:42.420390+0000","last_peered":"2024-08-15T16:59:42.420390+0000","last_clean":"2024-08-15T16:59:42.420390+0000","last_became_active":"2024-08-15T16:59:42.415630+0000","last_became_peered":"2024-08-15T16:59:42.415630+0000","last_unstale":"2024-08-15T16:59:42.420390+0000","last_undegraded":"2024-08-15T16:59:42.420390+0000","last_fullsized":"2024-08-15T16:59:42.420390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:13:23.043603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.420472+0000","last_change":"2024-08-15T16:59:41.379215+0000","last_active":"2024-08-15T16:59:42.420472+0000","last_peered":"2024-08-15T16:59:42.420472+0000","last_clean":"2024-08-15T16:59:42.420472+0000","last_became_active":"2024-08-15T16:59:41.378905+0000","last_became_peered":"2024-08-15T16:59:41.378905+0000","last_unstale":"2024-08-15T16:59:42.420472+0000","last_undegraded":"2024-08-15T16:59:42.420472+0000","last_fullsized":"2024-08-15T16:59:42.420472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:18:55.486882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.416924+0000","last_change":"2024-08-15T16:59:42.408126+0000","last_active":"2024-08-15T16:59:42.416924+0000","last_peered":"2024-08-15T16:59:42.416924+0000","last_clean":"2024-08-15T16:59:42.416924+0000","last_became_active":"2024-08-15T16:59:42.407836+0000","last_became_peered":"2024-08-15T16:59:42.407836+0000","last_unstale":"2024-08-15T16:59:42.416924+0000","last_undegraded":"2024-08-15T16:59:42.416924+0000","last_fullsized":"2024-08-15T16:59:42.416924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:41.349871+0000","last_clean_scrub_stamp":"2024-08-15T16:59:41.349871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:38:58.343891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-15T16:59:42.421366+0000","last_change":"2024-08-15T16:59:41.382335+0000","last_active":"2024-08-15T16:59:42.421366+0000","last_peered":"2024-08-15T16:59:42.421366+0000","last_clean":"2024-08-15T16:59:42.421366+0000","last_became_active":"2024-08-15T16:59:41.382004+0000","last_became_peered":"2024-08-15T16:59:41.382004+0000","last_unstale":"2024-08-15T16:59:42.421366+0000","last_undegraded":"2024-08-15T16:59:42.421366+0000","last_fullsized":"2024-08-15T16:59:42.421366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:40.339920+0000","last_clean_scrub_stamp":"2024-08-15T16:59:40.339920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:57:29.281887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298148,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073692,"statfs":{"total":96636764160,"available":96331460608,"internally_reserved":0,"allocated":811008,"data_stored":664727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":217088,"data_stored":73879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":225280,"data_stored":73915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1160000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":225280,"data_stored":75341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297564,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074276,"statfs":{"total":96636764160,"available":96332058624,"internally_reserved":0,"allocated":212992,"data_stored":73789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297576,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074264,"statfs":{"total":96636764160,"available":96332046336,"internally_reserved":0,"allocated":221184,"data_stored":73949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":663822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T16:59:52.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T16:59:52.499 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-15T16:59:52.499 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-15T16:59:52.499 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-15T16:59:52.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 Processor -- start 2024-08-15T16:59:52.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 -- start start 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f2c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f2c0af200 0x7f0f2c0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c0ae660 con 0x7f0f2c15d960 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c0c9f80 con 0x7f0f2c0af200 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f30c6c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b2fd0 con 0x7f0f2c15d590 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f29d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0f2c15d960 0x7f0f2c0aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60176/0 (socket says 172.21.15.146:60176) 2024-08-15T16:59:52.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f29d9b700 1 -- 172.21.15.146:0/2116619756 learned_addr learned my addr 172.21.15.146:0/2116619756 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:52.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f2c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2ad9d700 1 --2- 172.21.15.146:0/2116619756 >> v2:172.21.15.161:3300/0 conn(0x7f0f2c0af200 0x7f0f2c0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300991810 0 0) 0x7f0f2c0ae660 con 0x7f0f2c15d960 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f18003570 con 0x7f0f2c15d960 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 462046045 0 0) 0x7f0f18003570 con 0x7f0f2c15d960 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 msgr2=0x7f0f2c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f2c0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 >> v2:172.21.15.161:3300/0 conn(0x7f0f2c0af200 msgr2=0x7f0f2c0cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 --2- 172.21.15.146:0/2116619756 >> v2:172.21.15.161:3300/0 conn(0x7f0f2c0af200 0x7f0f2c0cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0c99c0 con 0x7f0f2c15d960 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f0f1c0017f0 con 0x7f0f2c15d960 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 >> v1:172.21.15.146:6789/0 conn(0x7f0f2c15d960 legacy=0x7f0f2c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f18003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f18004810 0x7f0f18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0c9f80 con 0x7f0f18004810 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1800ba60 con 0x7f0f18003a60 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1800bba0 con 0x7f0f2c15d590 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f18003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f29d9b700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f18004810 0x7f0f18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2ad9d700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 -- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 msgr2=0x7f0f180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c15d590 0x7f0f180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 -- 172.21.15.146:0/2116619756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f18004810 msgr2=0x7f0f18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f18004810 0x7f0f18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.585+0000 7f0f2a59c700 1 -- 172.21.15.146:0/2116619756 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f18003570 con 0x7f0f18003a60 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f18003e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009c30 tx=0x7f0f140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e480e0fc8065fb03 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400e710 con 0x7f0f18003a60 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1400e870 con 0x7f0f18003a60 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f2959a700 1 -- 172.21.15.146:0/2116619756 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400ec30 con 0x7f0f18003a60 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 msgr2=0x7f0f18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2116619756 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f18003e70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0f14009c30 tx=0x7f0f140091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f2ad9d700 1 -- 172.21.15.146:0/2116619756 reap_dead start 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2116619756 shutdown_connections 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2116619756 >> 172.21.15.146:0/2116619756 conn(0x7f0f2c0a9db0 msgr2=0x7f0f2c0ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2116619756 shutdown_connections 2024-08-15T16:59:52.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2116619756 wait complete. 2024-08-15T16:59:52.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 Processor -- start 2024-08-15T16:59:52.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.589+0000 7f0f30c6c700 1 -- start start 2024-08-15T16:59:52.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f2c170ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c16dd50 0x7f0f2c0987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c15a760 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c15aa40 con 0x7f0f18003a60 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c15ad20 con 0x7f0f2c16dd50 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49888/0 (socket says 172.21.15.146:49888) 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 -- 172.21.15.146:0/2286277936 learned_addr learned my addr 172.21.15.146:0/2286277936 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f2c170ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f29d9b700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c16dd50 0x7f0f2c0987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 -- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c16dd50 msgr2=0x7f0f2c0987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c16dd50 0x7f0f2c0987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 -- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 msgr2=0x7f0f2c170ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f2c170ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 -- 172.21.15.146:0/2286277936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c15b000 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f2ad9d700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006f00 tx=0x7f0f240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24013750 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240138d0 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c171010 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24013c90 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.593+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c098d00 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c15b1f0 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0f24014650 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f227fc700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 0x7f0f0c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f2407cf90 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 0x7f0f0c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.597+0000 7f0f2a59c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 0x7f0f0c062d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f140091e0 tx=0x7f0f1401a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.601+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0f2404d460 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.785+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0f2c15a8a0 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.799 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f227fc700 1 -- 172.21.15.146:0/2286277936 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0f24022040 con 0x7f0f2c16e1c0 2024-08-15T16:59:52.799 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:52.800 INFO:teuthology.orchestra.run.smithi146.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 msgr2=0x7f0f0c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 0x7f0f0c062d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f140091e0 tx=0x7f0f1401a000 comp rx=0 tx=0).stop 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 msgr2=0x7f0f2c099b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0f24006f00 tx=0x7f0f240091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 shutdown_connections 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f0c060860 0x7f0f0c062d10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.802 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f2c16e1c0 0x7f0f2c099b60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f2c16dd50 0x7f0f2c0987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 --2- 172.21.15.146:0/2286277936 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f18003a60 0x7f0f2c170ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 >> 172.21.15.146:0/2286277936 conn(0x7f0f2c0a9db0 msgr2=0x7f0f2c0aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 shutdown_connections 2024-08-15T16:59:52.803 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.789+0000 7f0f30c6c700 1 -- 172.21.15.146:0/2286277936 wait complete. 2024-08-15T16:59:52.813 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-15T16:59:52.813 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:52.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 Processor -- start 2024-08-15T16:59:52.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 -- start start 2024-08-15T16:59:52.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac740cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac7415d5b0 0x7fac740d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac740cf190 con 0x7fac740d0010 2024-08-15T16:59:52.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740d09a0 con 0x7fac7415d5b0 2024-08-15T16:59:52.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740ce5e0 con 0x7fac74159040 2024-08-15T16:59:52.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac72d9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fac7415d5b0 0x7fac740d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7259c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac740cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac69d9b700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fac740d0010 0x7fac7415cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42114/0 (socket says 172.21.15.146:42114) 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac69d9b700 1 -- 172.21.15.146:0/2186098856 learned_addr learned my addr 172.21.15.146:0/2186098856 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.901+0000 7fac7259c700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac740cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59788/0 (socket says 172.21.15.146:59788) 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac72d9d700 1 -- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 msgr2=0x7fac740cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac72d9d700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac740cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac72d9d700 1 -- 172.21.15.146:0/2186098856 >> v1:172.21.15.161:6789/0 conn(0x7fac740d0010 legacy=0x7fac7415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac72d9d700 1 -- 172.21.15.146:0/2186098856 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740ce860 con 0x7fac7415d5b0 2024-08-15T16:59:52.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac72d9d700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7415d5b0 0x7fac740d0460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009e60 tx=0x7fac6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278f241baedb2e53 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac71d9b700 1 -- 172.21.15.146:0/2186098856 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00e070 con 0x7fac7415d5b0 2024-08-15T16:59:52.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac71d9b700 1 -- 172.21.15.146:0/2186098856 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c00bb60 con 0x7fac7415d5b0 2024-08-15T16:59:52.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac71d9b700 1 -- 172.21.15.146:0/2186098856 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c012430 con 0x7fac7415d5b0 2024-08-15T16:59:52.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7415d5b0 msgr2=0x7fac740d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7415d5b0 0x7fac740d0460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009e60 tx=0x7fac6c009780 comp rx=0 tx=0).stop 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- 172.21.15.146:0/2186098856 shutdown_connections 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac7415d5b0 0x7fac740d0460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 --2- 172.21.15.146:0/2186098856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac740cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- 172.21.15.146:0/2186098856 >> 172.21.15.146:0/2186098856 conn(0x7fac74068250 msgr2=0x7fac74068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- 172.21.15.146:0/2186098856 shutdown_connections 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- 172.21.15.146:0/2186098856 wait complete. 2024-08-15T16:59:52.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 Processor -- start 2024-08-15T16:59:52.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.905+0000 7fac7359e700 1 -- start start 2024-08-15T16:59:52.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 0x7fac7416ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac74159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.921 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b0fc0 con 0x7fac7415d5b0 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b12a0 con 0x7fac740d0010 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b1580 con 0x7fac74159040 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49898/0 (socket says 172.21.15.146:49898) 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 -- 172.21.15.146:0/4041607875 learned_addr learned my addr 172.21.15.146:0/4041607875 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7259c700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 0x7fac7416ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac69d9b700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac74159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 -- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 msgr2=0x7fac74159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac74159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 -- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 msgr2=0x7fac7416ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 0x7fac7416ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 -- 172.21.15.146:0/4041607875 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c009a20 con 0x7fac7415d5b0 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7259c700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 0x7fac7416ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac69d9b700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac74159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac72d9d700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac74088ef0 tx=0x7fac6c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c01d630 con 0x7fac7415d5b0 2024-08-15T16:59:52.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c01d7b0 con 0x7fac7415d5b0 2024-08-15T16:59:52.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac740b1860 con 0x7fac7415d5b0 2024-08-15T16:59:52.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c01db70 con 0x7fac7415d5b0 2024-08-15T16:59:52.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.909+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7416b280 con 0x7fac7415d5b0 2024-08-15T16:59:52.924 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac740643a0 con 0x7fac7415d5b0 2024-08-15T16:59:52.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fac6c024070 con 0x7fac7415d5b0 2024-08-15T16:59:52.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac6affd700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 0x7fac58062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:52.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac6c07f400 con 0x7fac7415d5b0 2024-08-15T16:59:52.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac7259c700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 0x7fac58062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:52.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.913+0000 7fac7259c700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 0x7fac58062d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac64009d80 tx=0x7fac64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:52.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:52.925+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fac6c04f900 con 0x7fac7415d5b0 2024-08-15T16:59:53.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.085+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac740b5e30 con 0x7fac7415d5b0 2024-08-15T16:59:53.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.085+0000 7fac6affd700 1 -- 172.21.15.146:0/4041607875 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fac6c052f20 con 0x7fac7415d5b0 2024-08-15T16:59:53.099 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:53.099 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:53.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 msgr2=0x7fac58062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 0x7fac58062d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fac64009d80 tx=0x7fac64008040 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 msgr2=0x7fac74151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac74088ef0 tx=0x7fac6c019000 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 shutdown_connections 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fac58060860 0x7fac58062d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fac7415d5b0 0x7fac74151910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fac74159040 0x7fac74159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 --2- 172.21.15.146:0/4041607875 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fac740d0010 0x7fac7416ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 >> 172.21.15.146:0/4041607875 conn(0x7fac74068250 msgr2=0x7fac74152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 shutdown_connections 2024-08-15T16:59:53.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.089+0000 7fac7359e700 1 -- 172.21.15.146:0/4041607875 wait complete. 2024-08-15T16:59:53.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T16:59:53.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T16:59:53.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 Processor -- start 2024-08-15T16:59:53.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 -- start start 2024-08-15T16:59:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe30413f680 0x7fe3040aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3040afc70 con 0x7fe3040b0100 2024-08-15T16:59:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3040adc00 con 0x7fe30413f680 2024-08-15T16:59:53.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.209+0000 7fe30a71d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3040af060 con 0x7fe30413fa50 2024-08-15T16:59:53.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30971b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe30413f680 0x7fe3040aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:59814/0 (socket says 172.21.15.146:59814) 2024-08-15T16:59:53.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 -- 172.21.15.146:0/2359159715 learned_addr learned my addr 172.21.15.146:0/2359159715 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe2fbfff700 1 -- 172.21.15.146:0/2359159715 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801145692 0 0) 0x7fe3040afc70 con 0x7fe3040b0100 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe2fbfff700 1 -- 172.21.15.146:0/2359159715 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec003570 con 0x7fe3040b0100 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 -- 172.21.15.146:0/2359159715 >> v2:172.21.15.161:3300/0 conn(0x7fe30413f680 msgr2=0x7fe3040aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2359159715 >> v2:172.21.15.161:3300/0 conn(0x7fe30413f680 0x7fe3040aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 -- 172.21.15.146:0/2359159715 >> v1:172.21.15.161:6789/0 conn(0x7fe3040b0100 legacy=0x7fe30406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 -- 172.21.15.146:0/2359159715 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040af340 con 0x7fe30413fa50 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2359159715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe300009f20 tx=0x7fe300009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f013db270ee616ab server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe2fbfff700 1 -- 172.21.15.146:0/2359159715 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30000e070 con 0x7fe30413fa50 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe2fbfff700 1 -- 172.21.15.146:0/2359159715 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30000b970 con 0x7fe30413fa50 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe2fbfff700 1 -- 172.21.15.146:0/2359159715 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000092e0 con 0x7fe30413fa50 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 -- 172.21.15.146:0/2359159715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 msgr2=0x7fe3040ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2359159715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe300009f20 tx=0x7fe300009840 comp rx=0 tx=0).stop 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 -- 172.21.15.146:0/2359159715 shutdown_connections 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2359159715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ad6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2359159715 >> v2:172.21.15.161:3300/0 conn(0x7fe30413f680 0x7fe3040aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 -- 172.21.15.146:0/2359159715 >> 172.21.15.146:0/2359159715 conn(0x7fe3040678a0 msgr2=0x7fe304067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 -- 172.21.15.146:0/2359159715 shutdown_connections 2024-08-15T16:59:53.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.213+0000 7fe30a71d700 1 -- 172.21.15.146:0/2359159715 wait complete. 2024-08-15T16:59:53.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 Processor -- start 2024-08-15T16:59:53.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- start start 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 0x7fe304138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304159360 con 0x7fe3040b0100 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304167a90 con 0x7fe30413f680 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe304168b70 con 0x7fe30413fa50 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49910/0 (socket says 172.21.15.146:49910) 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 -- 172.21.15.146:0/2208656514 learned_addr learned my addr 172.21.15.146:0/2208656514 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:53.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 0x7fe304138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe309f1c700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 -- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 msgr2=0x7fe3040ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 -- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 msgr2=0x7fe304138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 0x7fe304138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 -- 172.21.15.146:0/2208656514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009ae0 con 0x7fe3040b0100 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 0x7fe304138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe309f1c700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30971b700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400ecb0 tx=0x7fe2f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400c9d0 con 0x7fe3040b0100 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f400cb30 con 0x7fe3040b0100 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400ced0 con 0x7fe3040b0100 2024-08-15T16:59:53.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe304158470 con 0x7fe3040b0100 2024-08-15T16:59:53.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.217+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe304158a30 con 0x7fe3040b0100 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe2f4013680 con 0x7fe3040b0100 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe304064330 con 0x7fe3040b0100 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe2f97fa700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 0x7fe2f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2f407cfe0 con 0x7fe3040b0100 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 0x7fe2f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.221+0000 7fe308f1a700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 0x7fe2f0062d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe30413fe20 tx=0x7fe300018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.229+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe2f404d3d0 con 0x7fe3040b0100 2024-08-15T16:59:53.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.381+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3040a9af0 con 0x7fe3040b0100 2024-08-15T16:59:53.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.381+0000 7fe2f97fa700 1 -- 172.21.15.146:0/2208656514 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2f40170a0 con 0x7fe3040b0100 2024-08-15T16:59:53.395 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T16:59:53.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 msgr2=0x7fe2f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 0x7fe2f0062d10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe30413fe20 tx=0x7fe300018000 comp rx=0 tx=0).stop 2024-08-15T16:59:53.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 msgr2=0x7fe304168630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400ecb0 tx=0x7fe2f40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 shutdown_connections 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe2f0060860 0x7fe2f0062d10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe30413fa50 0x7fe3040ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe30413f680 0x7fe304138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 --2- 172.21.15.146:0/2208656514 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe3040b0100 0x7fe304168630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 >> 172.21.15.146:0/2208656514 conn(0x7fe3040678a0 msgr2=0x7fe30413e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 shutdown_connections 2024-08-15T16:59:53.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.385+0000 7fe30a71d700 1 -- 172.21.15.146:0/2208656514 wait complete. 2024-08-15T16:59:53.408 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T16:59:53.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.505+0000 7fb68a05e700 1 Processor -- start 2024-08-15T16:59:53.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.505+0000 7fb68a05e700 1 -- start start 2024-08-15T16:59:53.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb68406ead0 0x7fb68406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6840b1830 0x7fb684155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406e010 0x7fb684152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6840ace30 con 0x7fb68406ead0 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6840ae6b0 con 0x7fb6840b1830 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb684156dd0 con 0x7fb68406e010 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb68406ead0 0x7fb68406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb68406ead0 0x7fb68406eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49912/0 (socket says 172.21.15.146:49912) 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 -- 172.21.15.146:0/2813276401 learned_addr learned my addr 172.21.15.146:0/2813276401 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68985d700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406e010 0x7fb684152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68885b700 1 --2- 172.21.15.146:0/2813276401 >> v2:172.21.15.161:3300/0 conn(0x7fb6840b1830 0x7fb684155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 -- 172.21.15.146:0/2813276401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406e010 msgr2=0x7fb684152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406e010 0x7fb684152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 -- 172.21.15.146:0/2813276401 >> v2:172.21.15.161:3300/0 conn(0x7fb6840b1830 msgr2=0x7fb684155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 --2- 172.21.15.146:0/2813276401 >> v2:172.21.15.161:3300/0 conn(0x7fb6840b1830 0x7fb684155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 -- 172.21.15.146:0/2813276401 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684156450 con 0x7fb68406ead0 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68905c700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406ead0 0x7fb68406eea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb68405f0b0 tx=0x7fb674009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39c73ba5a7456c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb67bfff700 1 -- 172.21.15.146:0/2813276401 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400b640 con 0x7fb68406ead0 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb67bfff700 1 -- 172.21.15.146:0/2813276401 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400b7a0 con 0x7fb68406ead0 2024-08-15T16:59:53.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb67bfff700 1 -- 172.21.15.146:0/2813276401 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400bb60 con 0x7fb68406ead0 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- 172.21.15.146:0/2813276401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406ead0 msgr2=0x7fb68406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406ead0 0x7fb68406eea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb68405f0b0 tx=0x7fb674009250 comp rx=0 tx=0).stop 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- 172.21.15.146:0/2813276401 shutdown_connections 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406e010 0x7fb684152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2813276401 >> v2:172.21.15.161:3300/0 conn(0x7fb6840b1830 0x7fb684155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2813276401 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406ead0 0x7fb68406eea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- 172.21.15.146:0/2813276401 >> 172.21.15.146:0/2813276401 conn(0x7fb6840678a0 msgr2=0x7fb684067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- 172.21.15.146:0/2813276401 shutdown_connections 2024-08-15T16:59:53.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.509+0000 7fb68a05e700 1 -- 172.21.15.146:0/2813276401 wait complete. 2024-08-15T16:59:53.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 Processor -- start 2024-08-15T16:59:53.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 -- start start 2024-08-15T16:59:53.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406ead0 0x7fb684150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6840b1830 0x7fb684166160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68414f290 con 0x7fb68406e010 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68414f570 con 0x7fb6840b1830 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68a05e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68414f850 con 0x7fb68406ead0 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.513+0000 7fb68905c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49924/0 (socket says 172.21.15.146:49924) 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 -- 172.21.15.146:0/2339962512 learned_addr learned my addr 172.21.15.146:0/2339962512 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68885b700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406ead0 0x7fb684150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68985d700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6840b1830 0x7fb684166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 -- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406ead0 msgr2=0x7fb684150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406ead0 0x7fb684150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 -- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6840b1830 msgr2=0x7fb684166160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6840b1830 0x7fb684166160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 -- 172.21.15.146:0/2339962512 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6740094f0 con 0x7fb68406e010 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68905c700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb684068b40 tx=0x7fb674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67400b910 con 0x7fb68406e010 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67400ba70 con 0x7fb68406e010 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68414fb30 con 0x7fb68406e010 2024-08-15T16:59:53.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674011630 con 0x7fb68406e010 2024-08-15T16:59:53.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb684147ef0 con 0x7fb68406e010 2024-08-15T16:59:53.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb684150030 con 0x7fb68406e010 2024-08-15T16:59:53.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.517+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb6740119a0 con 0x7fb68406e010 2024-08-15T16:59:53.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.521+0000 7fb6797fa700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 0x7fb668062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.521+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb67407e250 con 0x7fb68406e010 2024-08-15T16:59:53.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.521+0000 7fb68885b700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 0x7fb668062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.521+0000 7fb68885b700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 0x7fb668062d10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6840afe70 tx=0x7fb6800091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.525+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb67404e600 con 0x7fb68406e010 2024-08-15T16:59:53.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.677+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb684064330 con 0x7fb68406e010 2024-08-15T16:59:53.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb6797fa700 1 -- 172.21.15.146:0/2339962512 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb674051c20 con 0x7fb68406e010 2024-08-15T16:59:53.691 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T16:59:53.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 msgr2=0x7fb668062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 0x7fb668062d10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb6840afe70 tx=0x7fb6800091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 msgr2=0x7fb6841479b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb684068b40 tx=0x7fb674017000 comp rx=0 tx=0).stop 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 shutdown_connections 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb668060860 0x7fb668062d10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6840b1830 0x7fb684166160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb68406ead0 0x7fb684150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 --2- 172.21.15.146:0/2339962512 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb68406e010 0x7fb6841479b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 >> 172.21.15.146:0/2339962512 conn(0x7fb6840678a0 msgr2=0x7fb684154890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 shutdown_connections 2024-08-15T16:59:53.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.681+0000 7fb68a05e700 1 -- 172.21.15.146:0/2339962512 wait complete. 2024-08-15T16:59:53.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T16:59:53.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.797+0000 7fd6f1da4700 1 Processor -- start 2024-08-15T16:59:53.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.797+0000 7fd6f1da4700 1 -- start start 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6ec06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0b06c0 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0afab0 con 0x7fd6ec0af620 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0afd90 con 0x7fd6ec1555d0 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd6ec06f2b0 0x7fd6ec0b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60186/0 (socket says 172.21.15.146:60186) 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 -- 172.21.15.146:0/1693941640 learned_addr learned my addr 172.21.15.146:0/1693941640 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:53.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eb7fe700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6ec06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775180549 0 0) 0x7fd6ec0b06c0 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6cc003570 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 934886142 0 0) 0x7fd6cc003570 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 msgr2=0x7fd6ec06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6ec06ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 >> v1:172.21.15.161:6789/0 conn(0x7fd6ec0af620 legacy=0x7fd6ec1592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0b0070 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7fd6dc001470 con 0x7fd6ec06f2b0 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 >> v1:172.21.15.146:6789/0 conn(0x7fd6ec06f2b0 legacy=0x7fd6ec0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6cc003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6cc004010 0x7fd6cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc004d50 0x7fd6cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0afd90 con 0x7fd6cc004010 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc00bbf0 con 0x7fd6cc004d50 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc00bdb0 con 0x7fd6ec1555d0 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6cc004010 0x7fd6cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eb7fe700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6cc003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ebfff700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc004d50 0x7fd6cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 -- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 msgr2=0x7fd6cc003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6ec1555d0 0x7fd6cc003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 -- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc004d50 msgr2=0x7fd6cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6cc004d50 0x7fd6cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 -- 172.21.15.146:0/1693941640 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc003570 con 0x7fd6cc004010 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6cc004010 0x7fd6cc004810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007cc0 tx=0x7fd6dc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0102478675f0a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00cc30 con 0x7fd6cc004010 2024-08-15T16:59:53.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00cd90 con 0x7fd6cc004010 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ea7fc700 1 -- 172.21.15.146:0/1693941640 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc005b70 con 0x7fd6cc004010 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- 172.21.15.146:0/1693941640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6cc004010 msgr2=0x7fd6cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/1693941640 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6cc004010 0x7fd6cc004810 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007cc0 tx=0x7fd6dc001e50 comp rx=0 tx=0).stop 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6ebfff700 1 -- 172.21.15.146:0/1693941640 reap_dead start 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- 172.21.15.146:0/1693941640 shutdown_connections 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- 172.21.15.146:0/1693941640 >> 172.21.15.146:0/1693941640 conn(0x7fd6ec066e00 msgr2=0x7fd6ec067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- 172.21.15.146:0/1693941640 shutdown_connections 2024-08-15T16:59:53.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.801+0000 7fd6f1da4700 1 -- 172.21.15.146:0/1693941640 wait complete. 2024-08-15T16:59:53.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 Processor -- start 2024-08-15T16:59:53.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- start start 2024-08-15T16:59:53.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc004010 0x7fd6ec099450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 0x7fd6ec159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec158360 con 0x7fd6ec169880 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec16a090 con 0x7fd6ec1587f0 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec16a370 con 0x7fd6cc004010 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49942/0 (socket says 172.21.15.146:49942) 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 -- 172.21.15.146:0/248527774 learned_addr learned my addr 172.21.15.146:0/248527774 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eb7fe700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc004010 0x7fd6ec099450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6ebfff700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 0x7fd6ec159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 -- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc004010 msgr2=0x7fd6ec099450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc004010 0x7fd6ec099450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 -- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 msgr2=0x7fd6ec159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 0x7fd6ec159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 -- 172.21.15.146:0/248527774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec16a650 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6ebfff700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 0x7fd6ec159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6eaffd700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007c90 tx=0x7fd6dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00c980 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc0070d0 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec0ae2a0 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc007450 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.805+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec157f10 con 0x7fd6ec169880 2024-08-15T16:59:53.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0645c0 con 0x7fd6ec169880 2024-08-15T16:59:53.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd6dc011470 con 0x7fd6ec169880 2024-08-15T16:59:53.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6e37fe700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 0x7fd6d0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:53.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd6dc07e4c0 con 0x7fd6ec169880 2024-08-15T16:59:53.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6eb7fe700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 0x7fd6d0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:53.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.809+0000 7fd6eb7fe700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 0x7fd6d0062d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006f00 tx=0x7fd6d40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:53.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.817+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd6dc04e910 con 0x7fd6ec169880 2024-08-15T16:59:53.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6ec068790 con 0x7fd6ec169880 2024-08-15T16:59:53.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6e37fe700 1 -- 172.21.15.146:0/248527774 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6dc051f30 con 0x7fd6ec169880 2024-08-15T16:59:53.992 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T16:59:53.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 msgr2=0x7fd6d0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 0x7fd6d0062d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4006f00 tx=0x7fd6d40091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 msgr2=0x7fd6ec0add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc007c90 tx=0x7fd6dc017000 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 shutdown_connections 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd6d0060890 0x7fd6d0062d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6ec1587f0 0x7fd6ec159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6ec169880 0x7fd6ec0add60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 --2- 172.21.15.146:0/248527774 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6cc004010 0x7fd6ec099450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 >> 172.21.15.146:0/248527774 conn(0x7fd6ec066e00 msgr2=0x7fd6ec06def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 shutdown_connections 2024-08-15T16:59:53.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:53.981+0000 7fd6f1da4700 1 -- 172.21.15.146:0/248527774 wait complete. 2024-08-15T16:59:54.007 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:54.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 Processor -- start 2024-08-15T16:59:54.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 -- start start 2024-08-15T16:59:54.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b380b14a0 0x7f6b380cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b380cbaa0 con 0x7f6b38154900 2024-08-15T16:59:54.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b38157630 con 0x7f6b380cb390 2024-08-15T16:59:54.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b38068790 con 0x7f6b380b14a0 2024-08-15T16:59:54.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b2ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6b380b14a0 0x7f6b380cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3659c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6b38154900 0x7f6b380b0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60192/0 (socket says 172.21.15.146:60192) 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.093+0000 7f6b3659c700 1 -- 172.21.15.146:0/3873168881 learned_addr learned my addr 172.21.15.146:0/3873168881 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b2ffff700 1 -- 172.21.15.146:0/3873168881 >> v1:172.21.15.161:6789/0 conn(0x7f6b380cb390 legacy=0x7f6b38160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b35d9b700 1 -- 172.21.15.146:0/3873168881 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090805759 0 0) 0x7f6b380cbaa0 con 0x7f6b38154900 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b2ffff700 1 -- 172.21.15.146:0/3873168881 >> v1:172.21.15.146:6789/0 conn(0x7f6b38154900 legacy=0x7f6b380b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b2ffff700 1 -- 172.21.15.146:0/3873168881 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38157910 con 0x7f6b380b14a0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b2ffff700 1 --2- 172.21.15.146:0/3873168881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b380cae50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b200098e0 tx=0x7f6b200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f401694b416a15 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b35d9b700 1 -- 172.21.15.146:0/3873168881 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20009e80 con 0x7f6b380b14a0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b35d9b700 1 -- 172.21.15.146:0/3873168881 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000b860 con 0x7f6b380b14a0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b35d9b700 1 -- 172.21.15.146:0/3873168881 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bbe0 con 0x7f6b380b14a0 2024-08-15T16:59:54.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- 172.21.15.146:0/3873168881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 msgr2=0x7f6b380cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 --2- 172.21.15.146:0/3873168881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b380cae50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b200098e0 tx=0x7f6b200091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- 172.21.15.146:0/3873168881 shutdown_connections 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 --2- 172.21.15.146:0/3873168881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b380cae50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- 172.21.15.146:0/3873168881 >> 172.21.15.146:0/3873168881 conn(0x7f6b380678d0 msgr2=0x7f6b38067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- 172.21.15.146:0/3873168881 shutdown_connections 2024-08-15T16:59:54.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- 172.21.15.146:0/3873168881 wait complete. 2024-08-15T16:59:54.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 Processor -- start 2024-08-15T16:59:54.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- start start 2024-08-15T16:59:54.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 0x7f6b380a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b38154900 0x7f6b38153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38142240 con 0x7f6b380b14a0 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b38145410 con 0x7f6b380cb390 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.097+0000 7f6b3759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380a6b50 con 0x7f6b38154900 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49966/0 (socket says 172.21.15.146:49966) 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 -- 172.21.15.146:0/379068334 learned_addr learned my addr 172.21.15.146:0/379068334 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b2ffff700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 0x7f6b380a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b36d9d700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b38154900 0x7f6b38153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 -- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b38154900 msgr2=0x7f6b38153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b38154900 0x7f6b38153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 -- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 msgr2=0x7f6b380a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 0x7f6b380a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 -- 172.21.15.146:0/379068334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20009420 con 0x7f6b380b14a0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b2ffff700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 0x7f6b380a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3659c700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c810 tx=0x7f6b28007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28005c00 con 0x7f6b380b14a0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28005d60 con 0x7f6b380b14a0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b380a6e30 con 0x7f6b380b14a0 2024-08-15T16:59:54.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28007340 con 0x7f6b380b14a0 2024-08-15T16:59:54.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b380a73f0 con 0x7f6b380b14a0 2024-08-15T16:59:54.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.101+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b380643a0 con 0x7f6b380b14a0 2024-08-15T16:59:54.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.105+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6b2801b400 con 0x7f6b380b14a0 2024-08-15T16:59:54.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.105+0000 7f6b2e7fc700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 0x7f6b18062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.105+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b2807c610 con 0x7f6b380b14a0 2024-08-15T16:59:54.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.105+0000 7f6b2ffff700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 0x7f6b18062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.105+0000 7f6b2ffff700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 0x7f6b18062d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b380b18a0 tx=0x7f6b20017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.113+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6b2804cae0 con 0x7f6b380b14a0 2024-08-15T16:59:54.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.309+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b3806f5f0 con 0x7f6b380b14a0 2024-08-15T16:59:54.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.313+0000 7f6b2e7fc700 1 -- 172.21.15.146:0/379068334 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f6b28050100 con 0x7f6b380b14a0 2024-08-15T16:59:54.324 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:54.324 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 msgr2=0x7f6b18062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 0x7f6b18062d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b380b18a0 tx=0x7f6b20017000 comp rx=0 tx=0).stop 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 msgr2=0x7f6b38144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c810 tx=0x7f6b28007d90 comp rx=0 tx=0).stop 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 shutdown_connections 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6b18060890 0x7f6b18062d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6b38154900 0x7f6b38153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6b380cb390 0x7f6b380a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 --2- 172.21.15.146:0/379068334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b380b14a0 0x7f6b38144ed0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 >> 172.21.15.146:0/379068334 conn(0x7f6b380678d0 msgr2=0x7f6b3806da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 shutdown_connections 2024-08-15T16:59:54.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.317+0000 7f6b3759e700 1 -- 172.21.15.146:0/379068334 wait complete. 2024-08-15T16:59:54.330 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T16:59:54.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:54.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 Processor -- start 2024-08-15T16:59:54.449 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 -- start start 2024-08-15T16:59:54.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 0x7f23741536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23740cd1b0 0x7f237415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237415dbd0 con 0x7f237415a190 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740ce430 con 0x7f23740cd1b0 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2373fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237415d370 con 0x7f2374153320 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f23737fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23740cd1b0 0x7f237415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2372ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 0x7f23741536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f23727fc700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f237415a190 0x7f23740ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:42156/0 (socket says 172.21.15.146:42156) 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f23727fc700 1 -- 172.21.15.146:0/1527664898 learned_addr learned my addr 172.21.15.146:0/1527664898 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:54.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f23737fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f23740cd1b0 0x7f237415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49970/0 (socket says 172.21.15.146:49970) 2024-08-15T16:59:54.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2371ffb700 1 -- 172.21.15.146:0/1527664898 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725643674 0 0) 0x7f237415dbd0 con 0x7f237415a190 2024-08-15T16:59:54.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.437+0000 7f2371ffb700 1 -- 172.21.15.146:0/1527664898 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c003570 con 0x7f237415a190 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f23737fe700 1 -- 172.21.15.146:0/1527664898 >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 msgr2=0x7f23741536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f23737fe700 1 --2- 172.21.15.146:0/1527664898 >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 0x7f23741536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f23737fe700 1 -- 172.21.15.146:0/1527664898 >> v1:172.21.15.161:6789/0 conn(0x7f237415a190 legacy=0x7f23740ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f23737fe700 1 -- 172.21.15.146:0/1527664898 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237415a590 con 0x7f23740cd1b0 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2372ffd700 1 --2- 172.21.15.146:0/1527664898 >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 0x7f23741536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f23737fe700 1 --2- 172.21.15.146:0/1527664898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23740cd1b0 0x7f237415cc60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f23740d0cd0 tx=0x7f236c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32eae897ecedbb77 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2371ffb700 1 -- 172.21.15.146:0/1527664898 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00e070 con 0x7f23740cd1b0 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2371ffb700 1 -- 172.21.15.146:0/1527664898 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00bab0 con 0x7f23740cd1b0 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2371ffb700 1 -- 172.21.15.146:0/1527664898 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00be70 con 0x7f23740cd1b0 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 -- 172.21.15.146:0/1527664898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23740cd1b0 msgr2=0x7f237415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 --2- 172.21.15.146:0/1527664898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23740cd1b0 0x7f237415cc60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f23740d0cd0 tx=0x7f236c009780 comp rx=0 tx=0).stop 2024-08-15T16:59:54.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 -- 172.21.15.146:0/1527664898 shutdown_connections 2024-08-15T16:59:54.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 --2- 172.21.15.146:0/1527664898 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23740cd1b0 0x7f237415cc60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 --2- 172.21.15.146:0/1527664898 >> v2:172.21.15.161:3300/0 conn(0x7f2374153320 0x7f23741536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 -- 172.21.15.146:0/1527664898 >> 172.21.15.146:0/1527664898 conn(0x7f23740d00a0 msgr2=0x7f23740d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 -- 172.21.15.146:0/1527664898 shutdown_connections 2024-08-15T16:59:54.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.441+0000 7f2373fff700 1 -- 172.21.15.146:0/1527664898 wait complete. 2024-08-15T16:59:54.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 Processor -- start 2024-08-15T16:59:54.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 -- start start 2024-08-15T16:59:54.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740cd1b0 0x7f23740c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237415a190 0x7f2374168190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b3b20 con 0x7f2374153320 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740ca7f0 con 0x7f23740cd1b0 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2373fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740ca510 con 0x7f237415a190 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49978/0 (socket says 172.21.15.146:49978) 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 -- 172.21.15.146:0/1891628282 learned_addr learned my addr 172.21.15.146:0/1891628282 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f2372ffd700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740cd1b0 0x7f23740c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23737fe700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237415a190 0x7f2374168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 -- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237415a190 msgr2=0x7f2374168190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237415a190 0x7f2374168190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 -- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740cd1b0 msgr2=0x7f23740c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740cd1b0 0x7f23740c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 -- 172.21.15.146:0/1891628282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368009420 con 0x7f2374153320 2024-08-15T16:59:54.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f23727fc700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23640011e0 tx=0x7f2364007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364005b30 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.445+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364005c90 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c009a20 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364007320 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740b3580 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23740638f0 con 0x7f2374153320 2024-08-15T16:59:54.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2364007690 con 0x7f2374153320 2024-08-15T16:59:54.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f235affd700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 0x7f2360062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f236407bef0 con 0x7f2374153320 2024-08-15T16:59:54.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.449+0000 7f2372ffd700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 0x7f2360062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.453+0000 7f2372ffd700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 0x7f2360062d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2368009f40 tx=0x7f23680091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.457+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f236404c340 con 0x7f2374153320 2024-08-15T16:59:54.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.621+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23740cde20 con 0x7f2374153320 2024-08-15T16:59:54.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f235affd700 1 -- 172.21.15.146:0/1891628282 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f236404f960 con 0x7f2374153320 2024-08-15T16:59:54.636 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:54.636 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:54.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 msgr2=0x7f2360062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 0x7f2360062d10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2368009f40 tx=0x7f23680091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.638 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 msgr2=0x7f2374067030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f23640011e0 tx=0x7f2364007df0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 shutdown_connections 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2360060860 0x7f2360062d10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237415a190 0x7f2374168190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2374153320 0x7f2374067030 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 --2- 172.21.15.146:0/1891628282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740cd1b0 0x7f23740c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 >> 172.21.15.146:0/1891628282 conn(0x7f23740d00a0 msgr2=0x7f237415b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 shutdown_connections 2024-08-15T16:59:54.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.625+0000 7f2373fff700 1 -- 172.21.15.146:0/1891628282 wait complete. 2024-08-15T16:59:54.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T16:59:54.651 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T16:59:54.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 Processor -- start 2024-08-15T16:59:54.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 -- start start 2024-08-15T16:59:54.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fec980af190 0x7fec980b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec9806e840 0x7fec980b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec98140f70 con 0x7fec9806e470 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec98141250 con 0x7fec980af190 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.749+0000 7fec9cd15700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec98141530 con 0x7fec9806e840 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec977fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fec9806e470 0x7fec980aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60198/0 (socket says 172.21.15.146:60198) 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec977fe700 1 -- 172.21.15.146:0/2150279812 learned_addr learned my addr 172.21.15.146:0/2150279812 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 --2- 172.21.15.146:0/2150279812 >> v2:172.21.15.146:3300/0 conn(0x7fec980af190 0x7fec980b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec97fff700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec9806e840 0x7fec980b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec967fc700 1 -- 172.21.15.146:0/2150279812 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585384260 0 0) 0x7fec98140f70 con 0x7fec9806e470 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec967fc700 1 -- 172.21.15.146:0/2150279812 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec80003570 con 0x7fec9806e470 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 -- 172.21.15.146:0/2150279812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec9806e840 msgr2=0x7fec980b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec9806e840 0x7fec980b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 -- 172.21.15.146:0/2150279812 >> v1:172.21.15.146:6789/0 conn(0x7fec9806e470 legacy=0x7fec980aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 -- 172.21.15.146:0/2150279812 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec98141810 con 0x7fec980af190 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec96ffd700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec980af190 0x7fec980b0ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0098e0 tx=0x7fec8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec50617871209ade server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec967fc700 1 -- 172.21.15.146:0/2150279812 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c011630 con 0x7fec980af190 2024-08-15T16:59:54.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec967fc700 1 -- 172.21.15.146:0/2150279812 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c011790 con 0x7fec980af190 2024-08-15T16:59:54.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec967fc700 1 -- 172.21.15.146:0/2150279812 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c011b10 con 0x7fec980af190 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- 172.21.15.146:0/2150279812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec980af190 msgr2=0x7fec980b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec980af190 0x7fec980b0ef0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0098e0 tx=0x7fec8c0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- 172.21.15.146:0/2150279812 shutdown_connections 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec9806e840 0x7fec980b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 --2- 172.21.15.146:0/2150279812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec980af190 0x7fec980b0ef0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- 172.21.15.146:0/2150279812 >> 172.21.15.146:0/2150279812 conn(0x7fec980678a0 msgr2=0x7fec98067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- 172.21.15.146:0/2150279812 shutdown_connections 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- 172.21.15.146:0/2150279812 wait complete. 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 Processor -- start 2024-08-15T16:59:54.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.753+0000 7fec9cd15700 1 -- start start 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9806e470 0x7fec98156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec980af190 0x7fec98154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec980cb5e0 con 0x7fec9806e840 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98154640 con 0x7fec9806e470 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980cad40 con 0x7fec980af190 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:49998/0 (socket says 172.21.15.146:49998) 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 -- 172.21.15.146:0/4277372114 learned_addr learned my addr 172.21.15.146:0/4277372114 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec977fe700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9806e470 0x7fec98156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec97fff700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec980af190 0x7fec98154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 -- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec980af190 msgr2=0x7fec98154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec980af190 0x7fec98154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 -- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9806e470 msgr2=0x7fec98156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9806e470 0x7fec98156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 -- 172.21.15.146:0/4277372114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c009420 con 0x7fec9806e840 2024-08-15T16:59:54.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec96ffd700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec8c013000 tx=0x7fec8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c011900 con 0x7fec9806e840 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c011a80 con 0x7fec9806e840 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec980cb020 con 0x7fec9806e840 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c01a680 con 0x7fec9806e840 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec98157030 con 0x7fec9806e840 2024-08-15T16:59:54.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.757+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec981547d0 con 0x7fec9806e840 2024-08-15T16:59:54.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.761+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fec8c01aa30 con 0x7fec9806e840 2024-08-15T16:59:54.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.761+0000 7fec7f7fe700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 0x7fec84062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:54.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.761+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fec8c07e670 con 0x7fec9806e840 2024-08-15T16:59:54.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.761+0000 7fec977fe700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 0x7fec84062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:54.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.761+0000 7fec977fe700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 0x7fec84062d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec90005c50 tx=0x7fec90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:54.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.769+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fec8c04eb00 con 0x7fec9806e840 2024-08-15T16:59:54.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.933+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec98154410 con 0x7fec9806e840 2024-08-15T16:59:54.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec7f7fe700 1 -- 172.21.15.146:0/4277372114 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec8c0520e0 con 0x7fec9806e840 2024-08-15T16:59:54.948 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T16:59:54.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 msgr2=0x7fec84062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 0x7fec84062d40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec90005c50 tx=0x7fec90009040 comp rx=0 tx=0).stop 2024-08-15T16:59:54.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 msgr2=0x7fec980ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:54.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fec8c013000 tx=0x7fec8c017000 comp rx=0 tx=0).stop 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 shutdown_connections 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fec84060890 0x7fec84062d40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec980af190 0x7fec98154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fec9806e840 0x7fec980ca230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 --2- 172.21.15.146:0/4277372114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec9806e470 0x7fec98156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 >> 172.21.15.146:0/4277372114 conn(0x7fec980678a0 msgr2=0x7fec9815c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 shutdown_connections 2024-08-15T16:59:54.951 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:54.937+0000 7fec9cd15700 1 -- 172.21.15.146:0/4277372114 wait complete. 2024-08-15T16:59:54.961 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T16:59:55.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 Processor -- start 2024-08-15T16:59:55.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 -- start start 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f769c0686a0 0x7f769c068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0ae490 con 0x7f769c1552a0 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c0ae920 con 0x7f769c0686a0 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a170c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0af040 con 0x7f769c155670 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f769bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f769c1552a0 0x7f769c0b1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60204/0 (socket says 172.21.15.146:60204) 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f769bfff700 1 -- 172.21.15.146:0/871348171 learned_addr learned my addr 172.21.15.146:0/871348171 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f769b7fe700 1 --2- 172.21.15.146:0/871348171 >> v2:172.21.15.146:3300/0 conn(0x7f769c0686a0 0x7f769c068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.061+0000 7f76a0f0b700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769affd700 1 -- 172.21.15.146:0/871348171 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411151127 0 0) 0x7f769c0ae490 con 0x7f769c1552a0 2024-08-15T16:59:55.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769affd700 1 -- 172.21.15.146:0/871348171 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7688003570 con 0x7f769c1552a0 2024-08-15T16:59:55.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769b7fe700 1 -- 172.21.15.146:0/871348171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 msgr2=0x7f769c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769b7fe700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769b7fe700 1 -- 172.21.15.146:0/871348171 >> v1:172.21.15.146:6789/0 conn(0x7f769c1552a0 legacy=0x7f769c0b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769b7fe700 1 -- 172.21.15.146:0/871348171 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c0ad880 con 0x7f769c0686a0 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769b7fe700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c0686a0 0x7f769c068b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76840098e0 tx=0x7f76840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4b9a2db1c3f247 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769affd700 1 -- 172.21.15.146:0/871348171 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400c750 con 0x7f769c0686a0 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769affd700 1 -- 172.21.15.146:0/871348171 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768400c8b0 con 0x7f769c0686a0 2024-08-15T16:59:55.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f769affd700 1 -- 172.21.15.146:0/871348171 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768400cc50 con 0x7f769c0686a0 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 -- 172.21.15.146:0/871348171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c0686a0 msgr2=0x7f769c068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c0686a0 0x7f769c068b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76840098e0 tx=0x7f76840091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 -- 172.21.15.146:0/871348171 shutdown_connections 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 --2- 172.21.15.146:0/871348171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c0686a0 0x7f769c068b20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 -- 172.21.15.146:0/871348171 >> 172.21.15.146:0/871348171 conn(0x7f769c06e350 msgr2=0x7f769c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 -- 172.21.15.146:0/871348171 shutdown_connections 2024-08-15T16:59:55.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.065+0000 7f76a170c700 1 -- 172.21.15.146:0/871348171 wait complete. 2024-08-15T16:59:55.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 Processor -- start 2024-08-15T16:59:55.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- start start 2024-08-15T16:59:55.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f769c1552a0 0x7f769c0c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c152100 con 0x7f769c146f20 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c1660f0 con 0x7f769c1552a0 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0c44a0 con 0x7f769c155670 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50016/0 (socket says 172.21.15.146:50016) 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 -- 172.21.15.146:0/4158504778 learned_addr learned my addr 172.21.15.146:0/4158504778 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f769bfff700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f769c1552a0 0x7f769c0c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f769b7fe700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 -- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 msgr2=0x7f769c0a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c0a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 -- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f769c1552a0 msgr2=0x7f769c0c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f769c1552a0 0x7f769c0c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 -- 172.21.15.146:0/4158504778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684009420 con 0x7f769c146f20 2024-08-15T16:59:55.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a0f0b700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7694007f20 tx=0x7f7694009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400eb90 con 0x7f769c146f20 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769400ed10 con 0x7f769c146f20 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c0c4780 con 0x7f769c146f20 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694014670 con 0x7f769c146f20 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.069+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c0c4d40 con 0x7f769c146f20 2024-08-15T16:59:55.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.073+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c166260 con 0x7f769c146f20 2024-08-15T16:59:55.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.073+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f76940149e0 con 0x7f769c146f20 2024-08-15T16:59:55.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.073+0000 7f7693fff700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 0x7f767c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.077+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f769407d190 con 0x7f769c146f20 2024-08-15T16:59:55.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.077+0000 7f769bfff700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 0x7f767c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.077+0000 7f769bfff700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 0x7f767c062d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f768c007fd0 tx=0x7f768c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.094 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.081+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f769404d5e0 con 0x7f769c146f20 2024-08-15T16:59:55.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.241+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f769c0638f0 con 0x7f769c146f20 2024-08-15T16:59:55.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.241+0000 7f7693fff700 1 -- 172.21.15.146:0/4158504778 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7694050c00 con 0x7f769c146f20 2024-08-15T16:59:55.255 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 msgr2=0x7f767c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 0x7f767c062d10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f768c007fd0 tx=0x7f768c009040 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 msgr2=0x7f769c164f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7694007f20 tx=0x7f7694009860 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 shutdown_connections 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f767c060860 0x7f767c062d10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f769c146f20 0x7f769c164f60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f769c155670 0x7f769c0a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 --2- 172.21.15.146:0/4158504778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f769c1552a0 0x7f769c0c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 >> 172.21.15.146:0/4158504778 conn(0x7f769c06e350 msgr2=0x7f769c067c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 shutdown_connections 2024-08-15T16:59:55.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.245+0000 7f76a170c700 1 -- 172.21.15.146:0/4158504778 wait complete. 2024-08-15T16:59:55.269 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T16:59:55.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 Processor -- start 2024-08-15T16:59:55.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 -- start start 2024-08-15T16:59:55.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f596c0cbcc0 0x7f596c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f596c15d9f0 0x7f596c0c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0cc350 con 0x7f596c0b0390 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0ca760 con 0x7f596c15d9f0 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5972042700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0cd3f0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f596c15d9f0 0x7f596c0c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f596c15d9f0 0x7f596c0c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50032/0 (socket says 172.21.15.146:50032) 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f597083f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f596c0cbcc0 0x7f596c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 -- 172.21.15.146:0/3805829839 learned_addr learned my addr 172.21.15.146:0/3805829839 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 -- 172.21.15.146:0/3805829839 >> v2:172.21.15.161:3300/0 conn(0x7f596c0cbcc0 msgr2=0x7f596c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 --2- 172.21.15.146:0/3805829839 >> v2:172.21.15.161:3300/0 conn(0x7f596c0cbcc0 0x7f596c15d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 -- 172.21.15.146:0/3805829839 >> v1:172.21.15.161:6789/0 conn(0x7f596c0b0390 legacy=0x7f596c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.369+0000 7f5971841700 1 -- 172.21.15.146:0/3805829839 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0cc630 con 0x7f596c15d9f0 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5971841700 1 --2- 172.21.15.146:0/3805829839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c15d9f0 0x7f596c0c2c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f596c065680 tx=0x7f5964009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3c420cf53fad0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5963fff700 1 -- 172.21.15.146:0/3805829839 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400e070 con 0x7f596c15d9f0 2024-08-15T16:59:55.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5963fff700 1 -- 172.21.15.146:0/3805829839 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596400ba20 con 0x7f596c15d9f0 2024-08-15T16:59:55.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5963fff700 1 -- 172.21.15.146:0/3805829839 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640092e0 con 0x7f596c15d9f0 2024-08-15T16:59:55.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- 172.21.15.146:0/3805829839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c15d9f0 msgr2=0x7f596c0c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- 172.21.15.146:0/3805829839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c15d9f0 0x7f596c0c2c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f596c065680 tx=0x7f5964009840 comp rx=0 tx=0).stop 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- 172.21.15.146:0/3805829839 shutdown_connections 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- 172.21.15.146:0/3805829839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c15d9f0 0x7f596c0c2c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- 172.21.15.146:0/3805829839 >> v2:172.21.15.161:3300/0 conn(0x7f596c0cbcc0 0x7f596c15d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- 172.21.15.146:0/3805829839 >> 172.21.15.146:0/3805829839 conn(0x7f596c0cdee0 msgr2=0x7f596c0ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- 172.21.15.146:0/3805829839 shutdown_connections 2024-08-15T16:59:55.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- 172.21.15.146:0/3805829839 wait complete. 2024-08-15T16:59:55.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 Processor -- start 2024-08-15T16:59:55.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- start start 2024-08-15T16:59:55.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f596c0b0390 0x7f596c0c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 0x7f596c159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15f580 con 0x7f596c0cbcc0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15f860 con 0x7f596c0b0390 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.373+0000 7f5972042700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15fb40 con 0x7f596c15d9f0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50048/0 (socket says 172.21.15.146:50048) 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 -- 172.21.15.146:0/3591073395 learned_addr learned my addr 172.21.15.146:0/3591073395 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5971040700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f596c0b0390 0x7f596c0c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5971841700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 0x7f596c159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 -- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 msgr2=0x7f596c159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 0x7f596c159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 -- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f596c0b0390 msgr2=0x7f596c0c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f596c0b0390 0x7f596c0c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 -- 172.21.15.146:0/3591073395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009ae0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.389 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5971841700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 0x7f596c159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f597083f700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f596800ec80 tx=0x7f59680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800c9d0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800cb30 con 0x7f596c0cbcc0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c155a70 con 0x7f596c0cbcc0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968013470 con 0x7f596c0cbcc0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c157580 con 0x7f596c0cbcc0 2024-08-15T16:59:55.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c15f6c0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.377+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5968013820 con 0x7f596c0cbcc0 2024-08-15T16:59:55.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.381+0000 7f59617fa700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 0x7f5950062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.381+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f596807cf40 con 0x7f596c0cbcc0 2024-08-15T16:59:55.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.381+0000 7f5971040700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 0x7f5950062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.381+0000 7f5971040700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 0x7f5950062d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f596c0b1ae0 tx=0x7f595c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.389+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f596c15f6c0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.553+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f596c15f6c0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.553+0000 7f59617fa700 1 -- 172.21.15.146:0/3591073395 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f596c15f6c0 con 0x7f596c0cbcc0 2024-08-15T16:59:55.566 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 msgr2=0x7f5950062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 0x7f5950062d10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f596c0b1ae0 tx=0x7f595c009040 comp rx=0 tx=0).stop 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 msgr2=0x7f596c154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f596800ec80 tx=0x7f59680091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 shutdown_connections 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5950060860 0x7f5950062d10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f596c15d9f0 0x7f596c159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f596c0cbcc0 0x7f596c154c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 --2- 172.21.15.146:0/3591073395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f596c0b0390 0x7f596c0c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 >> 172.21.15.146:0/3591073395 conn(0x7f596c0cdee0 msgr2=0x7f596c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 shutdown_connections 2024-08-15T16:59:55.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.557+0000 7f5972042700 1 -- 172.21.15.146:0/3591073395 wait complete. 2024-08-15T16:59:55.585 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 Processor -- start 2024-08-15T16:59:55.686 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 -- start start 2024-08-15T16:59:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2360159040 0x7f23600cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 0x7f23600d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f236015cea0 0x7f236015d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23600cf190 con 0x7f236015cea0 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23600d09a0 con 0x7f2360159040 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.673+0000 7f2364a3f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600ce5e0 con 0x7f23600d0010 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f236015cea0 0x7f236015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f236015cea0 0x7f236015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50050/0 (socket says 172.21.15.146:50050) 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 -- 172.21.15.146:0/3227844075 learned_addr learned my addr 172.21.15.146:0/3227844075 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235effd700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 0x7f23600d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235f7fe700 1 --2- 172.21.15.146:0/3227844075 >> v2:172.21.15.161:3300/0 conn(0x7f2360159040 0x7f23600cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 -- 172.21.15.146:0/3227844075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 msgr2=0x7f23600d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 0x7f23600d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 -- 172.21.15.146:0/3227844075 >> v2:172.21.15.161:3300/0 conn(0x7f2360159040 msgr2=0x7f23600cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 --2- 172.21.15.146:0/3227844075 >> v2:172.21.15.161:3300/0 conn(0x7f2360159040 0x7f23600cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 -- 172.21.15.146:0/3227844075 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23600ce860 con 0x7f236015cea0 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235effd700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 0x7f23600d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235ffff700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236015cea0 0x7f236015d320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2360070d70 tx=0x7f2358009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd387c2b3a08430 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235e7fc700 1 -- 172.21.15.146:0/3227844075 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800e070 con 0x7f236015cea0 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235e7fc700 1 -- 172.21.15.146:0/3227844075 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235800b970 con 0x7f236015cea0 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f235e7fc700 1 -- 172.21.15.146:0/3227844075 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580092e0 con 0x7f236015cea0 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 -- 172.21.15.146:0/3227844075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236015cea0 msgr2=0x7f236015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236015cea0 0x7f236015d320 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2360070d70 tx=0x7f2358009840 comp rx=0 tx=0).stop 2024-08-15T16:59:55.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 -- 172.21.15.146:0/3227844075 shutdown_connections 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236015cea0 0x7f236015d320 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 --2- 172.21.15.146:0/3227844075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23600d0010 0x7f23600d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 --2- 172.21.15.146:0/3227844075 >> v2:172.21.15.161:3300/0 conn(0x7f2360159040 0x7f23600cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 -- 172.21.15.146:0/3227844075 >> 172.21.15.146:0/3227844075 conn(0x7f2360068250 msgr2=0x7f2360068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 -- 172.21.15.146:0/3227844075 shutdown_connections 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.677+0000 7f2364a3f700 1 -- 172.21.15.146:0/3227844075 wait complete. 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 Processor -- start 2024-08-15T16:59:55.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- start start 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2360159040 0x7f2360068620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 0x7f2360068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600a7180 con 0x7f23600d0010 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23600b2110 con 0x7f236015cea0 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600a6b30 con 0x7f2360159040 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50052/0 (socket says 172.21.15.146:50052) 2024-08-15T16:59:55.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 -- 172.21.15.146:0/2043597332 learned_addr learned my addr 172.21.15.146:0/2043597332 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235ffff700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 0x7f2360068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235effd700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2360159040 0x7f2360068620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 -- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2360159040 msgr2=0x7f2360068620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2360159040 0x7f2360068620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 -- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 msgr2=0x7f2360068b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 0x7f2360068b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 -- 172.21.15.146:0/2043597332 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350009420 con 0x7f23600d0010 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235ffff700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 0x7f2360068b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f235f7fe700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f235400ec80 tx=0x7f23540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400c9d0 con 0x7f23600d0010 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235400cb30 con 0x7f23600d0010 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2358009ae0 con 0x7f23600d0010 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2354013470 con 0x7f23600d0010 2024-08-15T16:59:55.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.681+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23601480b0 con 0x7f23600d0010 2024-08-15T16:59:55.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23600643a0 con 0x7f23600d0010 2024-08-15T16:59:55.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2354013820 con 0x7f23600d0010 2024-08-15T16:59:55.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f23477fe700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 0x7f234c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:55.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f235407cf10 con 0x7f23600d0010 2024-08-15T16:59:55.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f235effd700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 0x7f234c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:55.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.685+0000 7f235effd700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 0x7f234c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23500093f0 tx=0x7f2350008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:55.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.693+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f235404d360 con 0x7f23600d0010 2024-08-15T16:59:55.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.897+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23600cfdf0 con 0x7f23600d0010 2024-08-15T16:59:55.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.897+0000 7f23477fe700 1 -- 172.21.15.146:0/2043597332 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f2354017090 con 0x7f23600d0010 2024-08-15T16:59:55.910 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:55.910 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 msgr2=0x7f234c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 0x7f234c062d10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f23500093f0 tx=0x7f2350008040 comp rx=0 tx=0).stop 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 msgr2=0x7f2360147b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f235400ec80 tx=0x7f23540091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 shutdown_connections 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f234c060860 0x7f234c062d10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236015cea0 0x7f2360068b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2360159040 0x7f2360068620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 --2- 172.21.15.146:0/2043597332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23600d0010 0x7f2360147b70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 >> 172.21.15.146:0/2043597332 conn(0x7f2360068250 msgr2=0x7f23601579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 shutdown_connections 2024-08-15T16:59:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:55.901+0000 7f2364a3f700 1 -- 172.21.15.146:0/2043597332 wait complete. 2024-08-15T16:59:55.914 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T16:59:55.924 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T16:59:56.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 Processor -- start 2024-08-15T16:59:56.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 -- start start 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f46940b0d20 0x7f469406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46940b1800 0x7f4694141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4694141a00 con 0x7f469406e450 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469406e8a0 con 0x7f46940b1800 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4694140b10 con 0x7f46940b0d20 2024-08-15T16:59:56.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46940b1800 0x7f4694141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46940b1800 0x7f4694141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50066/0 (socket says 172.21.15.146:50066) 2024-08-15T16:59:56.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 -- 172.21.15.146:0/2545784094 learned_addr learned my addr 172.21.15.146:0/2545784094 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:56.030 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4698ee8700 1 --2- 172.21.15.146:0/2545784094 >> v2:172.21.15.161:3300/0 conn(0x7f46940b0d20 0x7f469406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 -- 172.21.15.146:0/2545784094 >> v2:172.21.15.161:3300/0 conn(0x7f46940b0d20 msgr2=0x7f469406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 --2- 172.21.15.146:0/2545784094 >> v2:172.21.15.161:3300/0 conn(0x7f46940b0d20 0x7f469406df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 -- 172.21.15.146:0/2545784094 >> v1:172.21.15.161:6789/0 conn(0x7f469406e450 legacy=0x7f46940b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 -- 172.21.15.146:0/2545784094 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4694140df0 con 0x7f46940b1800 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46996e9700 1 --2- 172.21.15.146:0/2545784094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f4694141e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f469406bd20 tx=0x7f4684009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7b5389e1689371 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46937fe700 1 -- 172.21.15.146:0/2545784094 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684009e70 con 0x7f46940b1800 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46937fe700 1 -- 172.21.15.146:0/2545784094 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468400b860 con 0x7f46940b1800 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f46937fe700 1 -- 172.21.15.146:0/2545784094 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400bc00 con 0x7f46940b1800 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 -- 172.21.15.146:0/2545784094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 msgr2=0x7f4694141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.017+0000 7f4699eea700 1 --2- 172.21.15.146:0/2545784094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f4694141e90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f469406bd20 tx=0x7f4684009250 comp rx=0 tx=0).stop 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 -- 172.21.15.146:0/2545784094 shutdown_connections 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 --2- 172.21.15.146:0/2545784094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f4694141e90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 --2- 172.21.15.146:0/2545784094 >> v2:172.21.15.161:3300/0 conn(0x7f46940b0d20 0x7f469406df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 -- 172.21.15.146:0/2545784094 >> 172.21.15.146:0/2545784094 conn(0x7f46940678d0 msgr2=0x7f4694067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:56.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 -- 172.21.15.146:0/2545784094 shutdown_connections 2024-08-15T16:59:56.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 -- 172.21.15.146:0/2545784094 wait complete. 2024-08-15T16:59:56.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 Processor -- start 2024-08-15T16:59:56.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.021+0000 7f4699eea700 1 -- start start 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469406e450 0x7f46941710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46940b0d20 0x7f4694171630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940a7e60 con 0x7f46940b1800 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46940a82f0 con 0x7f469406e450 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46940a7250 con 0x7f46940b0d20 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50080/0 (socket says 172.21.15.146:50080) 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 -- 172.21.15.146:0/160309895 learned_addr learned my addr 172.21.15.146:0/160309895 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4698ee8700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469406e450 0x7f46941710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4693fff700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46940b0d20 0x7f4694171630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 -- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46940b0d20 msgr2=0x7f4694171630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46940b0d20 0x7f4694171630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 -- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469406e450 msgr2=0x7f46941710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469406e450 0x7f46941710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 -- 172.21.15.146:0/160309895 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840094f0 con 0x7f46940b1800 2024-08-15T16:59:56.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f46996e9700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4684009290 tx=0x7f4684017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684025420 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840255a0 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940a7530 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684025960 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940a7af0 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.025+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46940a7fd0 con 0x7f46940b1800 2024-08-15T16:59:56.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.029+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4684025d50 con 0x7f46940b1800 2024-08-15T16:59:56.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.029+0000 7f4690ff9700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 0x7f4680062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.029+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f468407e6a0 con 0x7f46940b1800 2024-08-15T16:59:56.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.029+0000 7f4698ee8700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 0x7f4680062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.029+0000 7f4698ee8700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 0x7f4680062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4694145720 tx=0x7f4688008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.033+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f468404eb30 con 0x7f46940b1800 2024-08-15T16:59:56.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.197+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46940643a0 con 0x7f46940b1800 2024-08-15T16:59:56.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.197+0000 7f4690ff9700 1 -- 172.21.15.146:0/160309895 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f4684052110 con 0x7f46940b1800 2024-08-15T16:59:56.210 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:56.210 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T16:59:56.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 msgr2=0x7f4680062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 0x7f4680062d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4694145720 tx=0x7f4688008040 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 msgr2=0x7f46940a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4684009290 tx=0x7f4684017000 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 shutdown_connections 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4680060860 0x7f4680062d10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46940b1800 0x7f46940a5ca0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46940b0d20 0x7f4694171630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 --2- 172.21.15.146:0/160309895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f469406e450 0x7f46941710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 >> 172.21.15.146:0/160309895 conn(0x7f46940678d0 msgr2=0x7f469415cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 shutdown_connections 2024-08-15T16:59:56.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.201+0000 7f4699eea700 1 -- 172.21.15.146:0/160309895 wait complete. 2024-08-15T16:59:56.224 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T16:59:56.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 Processor -- start 2024-08-15T16:59:56.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 -- start start 2024-08-15T16:59:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f806df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46f80b1800 0x7f46f8141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f8141a00 con 0x7f46f806e450 2024-08-15T16:59:56.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f806e8a0 con 0x7f46f80b1800 2024-08-15T16:59:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8140b10 con 0x7f46f80b0d20 2024-08-15T16:59:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46f80b1800 0x7f46f8141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f46f80b1800 0x7f46f8141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50086/0 (socket says 172.21.15.146:50086) 2024-08-15T16:59:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 -- 172.21.15.146:0/3710278826 learned_addr learned my addr 172.21.15.146:0/3710278826 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:56.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f659c700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 -- 172.21.15.146:0/3710278826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 msgr2=0x7f46f806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f806df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 -- 172.21.15.146:0/3710278826 >> v1:172.21.15.161:6789/0 conn(0x7f46f806e450 legacy=0x7f46f80b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.321+0000 7f46f6d9d700 1 -- 172.21.15.146:0/3710278826 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8140df0 con 0x7f46f80b1800 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f6d9d700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f80b1800 0x7f46f8141e90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009f20 tx=0x7f46f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4246a6a951488f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f559a700 1 -- 172.21.15.146:0/3710278826 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000e070 con 0x7f46f80b1800 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f559a700 1 -- 172.21.15.146:0/3710278826 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000ba80 con 0x7f46f80b1800 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f559a700 1 -- 172.21.15.146:0/3710278826 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00092e0 con 0x7f46f80b1800 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- 172.21.15.146:0/3710278826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f80b1800 msgr2=0x7f46f8141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f80b1800 0x7f46f8141e90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009f20 tx=0x7f46f0009840 comp rx=0 tx=0).stop 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- 172.21.15.146:0/3710278826 shutdown_connections 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f80b1800 0x7f46f8141e90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 --2- 172.21.15.146:0/3710278826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f806df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- 172.21.15.146:0/3710278826 >> 172.21.15.146:0/3710278826 conn(0x7f46f80678d0 msgr2=0x7f46f8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- 172.21.15.146:0/3710278826 shutdown_connections 2024-08-15T16:59:56.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- 172.21.15.146:0/3710278826 wait complete. 2024-08-15T16:59:56.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 Processor -- start 2024-08-15T16:59:56.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.325+0000 7f46f759e700 1 -- start start 2024-08-15T16:59:56.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f8161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f80b1800 0x7f46f816dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f815b1c0 con 0x7f46f806e450 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81630f0 con 0x7f46f80b1800 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f813fd00 con 0x7f46f80b0d20 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50088/0 (socket says 172.21.15.146:50088) 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 -- 172.21.15.146:0/3716689331 learned_addr learned my addr 172.21.15.146:0/3716689331 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f6d9d700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f80b1800 0x7f46f816dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f5d9b700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f8161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 -- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 msgr2=0x7f46f8161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f8161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 -- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f80b1800 msgr2=0x7f46f816dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f80b1800 0x7f46f816dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 -- 172.21.15.146:0/3716689331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009ae0 con 0x7f46f806e450 2024-08-15T16:59:56.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f659c700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46e800d890 tx=0x7f46e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e80125f0 con 0x7f46f806e450 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e8012750 con 0x7f46f806e450 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8012af0 con 0x7f46f806e450 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f81624e0 con 0x7f46f806e450 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f8162aa0 con 0x7f46f806e450 2024-08-15T16:59:56.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.329+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f80643a0 con 0x7f46f806e450 2024-08-15T16:59:56.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.333+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f46e801e460 con 0x7f46f806e450 2024-08-15T16:59:56.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.333+0000 7f46ee7fc700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 0x7f46d8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:56.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.333+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46e807d110 con 0x7f46f806e450 2024-08-15T16:59:56.344 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.333+0000 7f46f5d9b700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 0x7f46d8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:56.345 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.333+0000 7f46f5d9b700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 0x7f46d8062d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46f8163ae0 tx=0x7f46e0007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:56.355 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.341+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f46e8050ff0 con 0x7f46f806e450 2024-08-15T16:59:56.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.545+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f8160910 con 0x7f46f806e450 2024-08-15T16:59:56.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46ee7fc700 1 -- 172.21.15.146:0/3716689331 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f46e8050b80 con 0x7f46f806e450 2024-08-15T16:59:56.559 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T16:59:56.560 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T16:59:56.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 msgr2=0x7f46d8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 0x7f46d8062d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f46f8163ae0 tx=0x7f46e0007ec0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 msgr2=0x7f46f813f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:56.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46e800d890 tx=0x7f46e8008040 comp rx=0 tx=0).stop 2024-08-15T16:59:56.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 shutdown_connections 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f46d8060890 0x7f46d8062d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f46f80b1800 0x7f46f816dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f46f80b0d20 0x7f46f8161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 --2- 172.21.15.146:0/3716689331 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f46f806e450 0x7f46f813f7c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 >> 172.21.15.146:0/3716689331 conn(0x7f46f80678d0 msgr2=0x7f46f815b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 shutdown_connections 2024-08-15T16:59:56.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:56.549+0000 7f46f759e700 1 -- 172.21.15.146:0/3716689331 wait complete. 2024-08-15T16:59:56.564 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T16:59:56.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T16:59:41.727299+0000', 'modified': '2024-08-15T16:59:44.405527+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/395806260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 395806260}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 395806260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T16:59:56.977 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T16:59:56.977 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-08-15T16:59:57.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.081+0000 7f5be2e09700 1 Processor -- start 2024-08-15T16:59:57.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.081+0000 7f5be2e09700 1 -- start start 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be2e09700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5bdc157760 0x7f5bdc0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be2e09700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5bdc0c4270 0x7f5bdc160d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be2e09700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0ca930 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be2e09700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0c8cd0 con 0x7f5bdc157760 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be2e09700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc0c2e10 con 0x7f5bdc0c4270 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5bdc157760 0x7f5bdc0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5bdc157760 0x7f5bdc0c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50096/0 (socket says 172.21.15.146:50096) 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3321382705 learned_addr learned my addr 172.21.15.146:0/3321382705 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be13a6700 1 --2- 172.21.15.146:0/3321382705 >> v2:172.21.15.161:3300/0 conn(0x7f5bdc0c4270 0x7f5bdc160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3321382705 >> v2:172.21.15.161:3300/0 conn(0x7f5bdc0c4270 msgr2=0x7f5bdc160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3321382705 >> v2:172.21.15.161:3300/0 conn(0x7f5bdc0c4270 0x7f5bdc160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3321382705 >> v1:172.21.15.161:6789/0 conn(0x7f5bdc0ca1a0 legacy=0x7f5bdc157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3321382705 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0cc500 con 0x7f5bdc157760 2024-08-15T16:59:57.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3321382705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc157760 0x7f5bdc0c8790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc06f250 tx=0x7f5bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49be3f5c2b6bb6d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5bdb7fe700 1 -- 172.21.15.146:0/3321382705 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4009e70 con 0x7f5bdc157760 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5bdb7fe700 1 -- 172.21.15.146:0/3321382705 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc400b860 con 0x7f5bdc157760 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.085+0000 7f5bdb7fe700 1 -- 172.21.15.146:0/3321382705 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400bc00 con 0x7f5bdc157760 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- 172.21.15.146:0/3321382705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc157760 msgr2=0x7f5bdc0c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 --2- 172.21.15.146:0/3321382705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc157760 0x7f5bdc0c8790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc06f250 tx=0x7f5bc4009250 comp rx=0 tx=0).stop 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- 172.21.15.146:0/3321382705 shutdown_connections 2024-08-15T16:59:57.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 --2- 172.21.15.146:0/3321382705 >> v2:172.21.15.161:3300/0 conn(0x7f5bdc0c4270 0x7f5bdc160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 --2- 172.21.15.146:0/3321382705 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc157760 0x7f5bdc0c8790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- 172.21.15.146:0/3321382705 >> 172.21.15.146:0/3321382705 conn(0x7f5bdc0caae0 msgr2=0x7f5bdc0caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- 172.21.15.146:0/3321382705 shutdown_connections 2024-08-15T16:59:57.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- 172.21.15.146:0/3321382705 wait complete. 2024-08-15T16:59:57.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 Processor -- start 2024-08-15T16:59:57.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.089+0000 7f5be2e09700 1 -- start start 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5bdc0c4270 0x7f5bdc154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bdc157760 0x7f5bdc0af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b3c10 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b3630 con 0x7f5bdc157760 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b1be0 con 0x7f5bdc0c4270 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50112/0 (socket says 172.21.15.146:50112) 2024-08-15T16:59:57.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 -- 172.21.15.146:0/2447914998 learned_addr learned my addr 172.21.15.146:0/2447914998 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be13a6700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bdc157760 0x7f5bdc0af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5bdc0c4270 0x7f5bdc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 -- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5bdc0c4270 msgr2=0x7f5bdc154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5bdc0c4270 0x7f5bdc154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 -- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bdc157760 msgr2=0x7f5bdc0af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bdc157760 0x7f5bdc0af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 -- 172.21.15.146:0/2447914998 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40094f0 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c4e0 tx=0x7f5bcc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bd8ff9700 1 -- 172.21.15.146:0/2447914998 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011590 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bd8ff9700 1 -- 172.21.15.146:0/2447914998 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc011710 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0b38a0 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5bd8ff9700 1 -- 172.21.15.146:0/2447914998 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011ad0 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.093+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0b1ec0 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5bd8ff9700 1 -- 172.21.15.146:0/2447914998 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5bcc01b3b0 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5bd8ff9700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5bd8ff9700 1 -- 172.21.15.146:0/2447914998 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bcc07bf20 con 0x7f5bdc0ca1a0 2024-08-15T16:59:57.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 Processor -- start 2024-08-15T16:59:57.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 -- start start 2024-08-15T16:59:57.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 0x7f5bbc062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc06f250 tx=0x7f5bc4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x13bb710 0x15ef710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 0x15f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1567ba0 con 0x13bbae0 2024-08-15T16:59:57.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1563b40 con 0x1406790 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x15290a0 con 0x13bb710 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be13a6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 0x15f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50122/0 (socket says 172.21.15.146:50122) 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3396058752 learned_addr learned my addr 172.21.15.146:0/3396058752 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x13bb710 0x15ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x13bb710 msgr2=0x15ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x13bb710 0x15ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 msgr2=0x15f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 0x15f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be0ba5700 1 -- 172.21.15.146:0/3396058752 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x14079e0 con 0x13bbae0 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.097+0000 7f5be13a6700 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 0x15f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:57.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b3c0 tx=0x7f5bc4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bec49151bcb68c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5bd1ffb700 1 -- 172.21.15.146:0/3396058752 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4031070 con 0x13bbae0 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5bd1ffb700 1 -- 172.21.15.146:0/3396058752 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc402cc50 con 0x13bbae0 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5bd1ffb700 1 -- 172.21.15.146:0/3396058752 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4021070 con 0x13bbae0 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- 172.21.15.146:0/3396058752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 msgr2=0x15ef1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b3c0 tx=0x7f5bc4021000 comp rx=0 tx=0).stop 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- 172.21.15.146:0/3396058752 shutdown_connections 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1406790 0x15f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x13bb710 0x15ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3396058752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x15ef1d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- 172.21.15.146:0/3396058752 >> 172.21.15.146:0/3396058752 conn(0x13bbeb0 msgr2=0x15f50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- 172.21.15.146:0/3396058752 shutdown_connections 2024-08-15T16:59:57.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- 172.21.15.146:0/3396058752 wait complete. 2024-08-15T16:59:57.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 -- start start 2024-08-15T16:59:57.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.101+0000 7f5be50b8740 1 Processor -- start 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bbae0 0x163cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x15f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1528d40 con 0x13bb710 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1501970 con 0x13bbae0 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1534690 con 0x1406790 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 10 client.0 _ll_register_callbacks cb 0x1505f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50124/0 (socket says 172.21.15.146:50124) 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 -- 172.21.15.146:0/2023292999 learned_addr learned my addr 172.21.15.146:0/2023292999 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be13a6700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x15f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bbae0 0x163cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 -- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 msgr2=0x15f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x15f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 -- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bbae0 msgr2=0x163cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bbae0 0x163cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 -- 172.21.15.146:0/2023292999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd4011230 con 0x13bb710 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4003080 tx=0x7f5bc402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6369c873df8269 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-15T16:59:57.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4031070 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bc403fad0 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13c9260 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bc4021040 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 10 client.4602 fetch_fsmap learned FSMap version 6 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7f5bdc0cc500 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13c9260 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5bc402a7a0 con 0x13bb710 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 10 client.4602 fetch_fsmap finished waiting for FSMap version 6 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 10 client.4602 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 20 client.4602 populate_metadata read hostname 'smithi146' 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1645a20 0x1647e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 4 client.4602 mds_command: new command op to 4245 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 --> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x14134e0 con 0x1645a20 2024-08-15T16:59:57.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.105+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1645a20 0x1647e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5bbaffd700 10 client.4602 ms_handle_connect on v2:172.21.15.161:6834/395806260 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1645a20 0x1647e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c18c3dfab64470ee server_cookie=325cfecb25443ea3 in_seq=0 out_seq=0 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5bbaffd700 1 -- 172.21.15.146:0/2023292999 <== mds.0 v2:172.21.15.161:6834/395806260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x14134e0 con 0x1645a20 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5bbaffd700 10 client.4602 handle_command_reply: tid=0 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 client.4602 shutdown 2024-08-15T16:59:57.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 msgr2=0x163c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4003080 tx=0x7f5bc402e050 comp rx=0 tx=0).stop 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 shutdown_connections 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x15f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bbae0 0x163cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1645a20 0x1647e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 --2- 172.21.15.146:0/2023292999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bb710 0x163c890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 >> 172.21.15.146:0/2023292999 conn(0x13bbeb0 msgr2=0x15f0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5bbaffd700 0 client.4602 ms_handle_reset on v2:172.21.15.161:6834/395806260 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 shutdown_connections 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 1 -- 172.21.15.146:0/2023292999 wait complete. 2024-08-15T16:59:57.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.109+0000 7f5be50b8740 20 client.4602 trim_cache size 0 max 16384 2024-08-15T16:59:57.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 Processor -- start 2024-08-15T16:59:57.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 -- start start 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x1690710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1691190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1512140 con 0x13bbae0 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x153b570 con 0x13bb710 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1567640 con 0x1406790 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50126/0 (socket says 172.21.15.146:50126) 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 -- 172.21.15.146:0/4017336553 learned_addr learned my addr 172.21.15.146:0/4017336553 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x1690710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be13a6700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1691190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 -- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 msgr2=0x1691190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1691190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 -- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 msgr2=0x1690710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x1690710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 -- 172.21.15.146:0/4017336553 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1531370 con 0x13bbae0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5be13a6700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1691190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c4b0 tx=0x7f5bcc063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba0d88deb8cfb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bd27fc700 1 -- 172.21.15.146:0/4017336553 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011e80 con 0x13bbae0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bd27fc700 1 -- 172.21.15.146:0/4017336553 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc07a550 con 0x13bbae0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.129+0000 7f5bd27fc700 1 -- 172.21.15.146:0/4017336553 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc050030 con 0x13bbae0 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/4017336553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 msgr2=0x1690c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c4b0 tx=0x7f5bcc063040 comp rx=0 tx=0).stop 2024-08-15T16:59:57.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/4017336553 shutdown_connections 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1691190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1690c50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- 172.21.15.146:0/4017336553 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x1690710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/4017336553 >> 172.21.15.146:0/4017336553 conn(0x13bbeb0 msgr2=0x1642fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/4017336553 shutdown_connections 2024-08-15T16:59:57.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/4017336553 wait complete. 2024-08-15T16:59:57.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- start start 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 Processor -- start 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x16944e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1648060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1599bb0 con 0x13bbae0 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x150c7c0 con 0x13bb710 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1569da0 con 0x1406790 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 10 client.0 _ll_register_callbacks cb 0x13879d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50128/0 (socket says 172.21.15.146:50128) 2024-08-15T16:59:57.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 -- 172.21.15.146:0/3115153962 learned_addr learned my addr 172.21.15.146:0/3115153962 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be13a6700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1648060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x16944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 -- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 msgr2=0x1648060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1648060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 -- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 msgr2=0x16944e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x16944e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 -- 172.21.15.146:0/3115153962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0b1ec0 con 0x13bbae0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bdbfff700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc04e380 tx=0x7f5bcc003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3534ac8955b904d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc057010 con 0x13bbae0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5bcc07a170 con 0x13bbae0 2024-08-15T16:59:57.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.133+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc402cc50 con 0x13bbae0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bcc06a250 con 0x13bbae0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 10 client.4608 fetch_fsmap learned FSMap version 6 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x1517d80 con 0x13bbae0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc06e080 con 0x13bbae0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5bcc04f520 con 0x13bbae0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 10 client.4608 fetch_fsmap finished waiting for FSMap version 6 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 10 client.4608 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 20 client.4608 populate_metadata read hostname 'smithi146' 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1648a20 0x1648df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 4 client.4608 mds_command: new command op to 4245 tid=0[{"prefix": "status"}] 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 --> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1438260 con 0x1648a20 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1648a20 0x1648df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 10 client.4608 ms_handle_connect on v2:172.21.15.161:6834/395806260 2024-08-15T16:59:57.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be0ba5700 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1648a20 0x1648df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48c497e5a2b6e65 server_cookie=35e70efa51753e80 in_seq=0 out_seq=0 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 1 -- 172.21.15.146:0/3115153962 <== mds.0 v2:172.21.15.161:6834/395806260 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x1438260 con 0x1648a20 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 10 client.4608 handle_command_reply: tid=0 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 client.4608 shutdown 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 msgr2=0x1694a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc04e380 tx=0x7f5bcc003000 comp rx=0 tx=0).stop 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 shutdown_connections 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1406790 0x1648060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:6834/395806260,v1:172.21.15.161:6835/395806260] conn(0x1648a20 0x1648df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x13bbae0 0x1694a20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 --2- 172.21.15.146:0/3115153962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x13bb710 0x16944e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5bbaffd700 0 client.4608 ms_handle_reset on v2:172.21.15.161:6834/395806260 2024-08-15T16:59:57.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 >> 172.21.15.146:0/3115153962 conn(0x13bbeb0 msgr2=0x15f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 shutdown_connections 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 1 -- 172.21.15.146:0/3115153962 wait complete. 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be50b8740 20 client.4608 trim_cache size 0 max 16384 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "10958773-fd73-437e-aa56-f078e9b084bc", 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 0, 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4245, 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-08-15T16:59:57.155 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 14.736234043, 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 6, 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 21, 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 21, 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 21.315781381000001 2024-08-15T16:59:57.156 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 msgr2=0x7f5bbc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 0x7f5bbc062d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc06f250 tx=0x7f5bc4017000 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 msgr2=0x7f5bdc0b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c4e0 tx=0x7f5bcc00c7f0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 shutdown_connections 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5bbc060860 0x7f5bbc062d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5bdc157760 0x7f5bdc0af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bdc0ca1a0 0x7f5bdc0b0e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 --2- 172.21.15.146:0/2447914998 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5bdc0c4270 0x7f5bdc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 >> 172.21.15.146:0/2447914998 conn(0x7f5bdc0caae0 msgr2=0x7f5bdc155090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 shutdown_connections 2024-08-15T16:59:57.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.137+0000 7f5be2e09700 1 -- 172.21.15.146:0/2447914998 wait complete. 2024-08-15T16:59:57.165 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-15T16:59:57.603 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-15T16:59:57.603 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-08-15T16:59:57.604 INFO:tasks.ceph_fuse:client.0 config is {} 2024-08-15T16:59:57.604 INFO:tasks.ceph_fuse:client.2 config is {} 2024-08-15T16:59:57.604 INFO:tasks.ceph_fuse:client.1 config is {} 2024-08-15T16:59:57.604 INFO:tasks.ceph_fuse:client.3 config is {} 2024-08-15T16:59:57.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:57.604 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-08-15T16:59:57.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:57.613 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-08-15T16:59:57.634 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-08-15T16:59:57.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:57.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:57.636 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-08-15T16:59:57.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:57.644 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-08-15T16:59:57.706 INFO:teuthology.orchestra.run.smithi146.stderr:Cannot find device "ceph-brx" 2024-08-15T16:59:57.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T16:59:57.708 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-15T16:59:57.708 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T16:59:57.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 Processor -- start 2024-08-15T16:59:57.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 -- start start 2024-08-15T16:59:57.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc156f80 0x7f08dc0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 0x7f08dc157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08dc157890 0x7f08dc0aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc0adf50 con 0x7f08dc157890 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc0ae920 con 0x7f08dc0ace80 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e39e3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0ad340 con 0x7f08dc156f80 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e1f80700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08dc157890 0x7f08dc0aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e1f80700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f08dc157890 0x7f08dc0aab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50142/0 (socket says 172.21.15.146:50142) 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e1f80700 1 -- 172.21.15.146:0/2786655034 learned_addr learned my addr 172.21.15.146:0/2786655034 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e0f7e700 1 --2- 172.21.15.146:0/2786655034 >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 0x7f08dc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.809+0000 7f08e177f700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc156f80 0x7f08dc0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 -- 172.21.15.146:0/2786655034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc156f80 msgr2=0x7f08dc0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc156f80 0x7f08dc0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 -- 172.21.15.146:0/2786655034 >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 msgr2=0x7f08dc157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 --2- 172.21.15.146:0/2786655034 >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 0x7f08dc157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 -- 172.21.15.146:0/2786655034 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0ad620 con 0x7f08dc157890 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e0f7e700 1 --2- 172.21.15.146:0/2786655034 >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 0x7f08dc157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e1f80700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc0aab70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08dc06c970 tx=0x7f08d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe331afb21c8cdb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08d3fff700 1 -- 172.21.15.146:0/2786655034 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400e070 con 0x7f08dc157890 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08d3fff700 1 -- 172.21.15.146:0/2786655034 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d400b970 con 0x7f08dc157890 2024-08-15T16:59:57.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08d3fff700 1 -- 172.21.15.146:0/2786655034 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d40092e0 con 0x7f08dc157890 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- 172.21.15.146:0/2786655034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 msgr2=0x7f08dc0aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc0aab70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f08dc06c970 tx=0x7f08d4009840 comp rx=0 tx=0).stop 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- 172.21.15.146:0/2786655034 shutdown_connections 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc0aab70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 --2- 172.21.15.146:0/2786655034 >> v2:172.21.15.161:3300/0 conn(0x7f08dc0ace80 0x7f08dc157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 --2- 172.21.15.146:0/2786655034 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc156f80 0x7f08dc0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- 172.21.15.146:0/2786655034 >> 172.21.15.146:0/2786655034 conn(0x7f08dc066e00 msgr2=0x7f08dc067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- 172.21.15.146:0/2786655034 shutdown_connections 2024-08-15T16:59:57.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- 172.21.15.146:0/2786655034 wait complete. 2024-08-15T16:59:57.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 Processor -- start 2024-08-15T16:59:57.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.813+0000 7f08e39e3700 1 -- start start 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 0x7f08dc150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 0x7f08dc14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc150a80 con 0x7f08dc157890 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc14feb0 con 0x7f08dc156f80 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc150150 con 0x7f08dc0ace80 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e177f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 0x7f08dc150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e0f7e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 0x7f08dc14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50144/0 (socket says 172.21.15.146:50144) 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e0f7e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 0x7f08dc14ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:34566/0 (socket says 172.21.15.146:34566) 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e177f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 0x7f08dc150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:60054/0 (socket says 172.21.15.146:60054) 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 -- 172.21.15.146:0/1695567167 learned_addr learned my addr 172.21.15.146:0/1695567167 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 -- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 msgr2=0x7f08dc150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 0x7f08dc150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 -- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 msgr2=0x7f08dc14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 0x7f08dc14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 -- 172.21.15.146:0/1695567167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8009420 con 0x7f08dc157890 2024-08-15T16:59:57.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e1f80700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08d4012000 tx=0x7f08d4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4007ac0 con 0x7f08dc157890 2024-08-15T16:59:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d4007c40 con 0x7f08dc157890 2024-08-15T16:59:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d401d420 con 0x7f08dc157890 2024-08-15T16:59:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d4009ae0 con 0x7f08dc157890 2024-08-15T16:59:57.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.817+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc150710 con 0x7f08dc157890 2024-08-15T16:59:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc0645c0 con 0x7f08dc157890 2024-08-15T16:59:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f08d4003040 con 0x7f08dc157890 2024-08-15T16:59:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08d17fa700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 0x7f08c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T16:59:57.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08d407e100 con 0x7f08dc157890 2024-08-15T16:59:57.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08e177f700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 0x7f08c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T16:59:57.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.821+0000 7f08e177f700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 0x7f08c0062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009db0 tx=0x7f08cc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T16:59:57.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:57.825+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f08d404e550 con 0x7f08dc157890 2024-08-15T16:59:58.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.025+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f08dc150bc0 con 0x7f08dc157890 2024-08-15T16:59:58.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.025+0000 7f08d17fa700 1 -- 172.21.15.146:0/1695567167 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f08d4051b70 con 0x7f08dc157890 2024-08-15T16:59:58.039 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 msgr2=0x7f08c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 0x7f08c0062d10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009db0 tx=0x7f08cc0091a0 comp rx=0 tx=0).stop 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 msgr2=0x7f08dc099290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f08d4012000 tx=0x7f08d4018000 comp rx=0 tx=0).stop 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 shutdown_connections 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f08c0060860 0x7f08c0062d10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f08dc157890 0x7f08dc099290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:58.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08dc156f80 0x7f08dc14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:58.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 --2- 172.21.15.146:0/1695567167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08dc0ace80 0x7f08dc150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T16:59:58.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 >> 172.21.15.146:0/1695567167 conn(0x7f08dc066e00 msgr2=0x7f08dc0aad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T16:59:58.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 shutdown_connections 2024-08-15T16:59:58.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T16:59:58.029+0000 7f08e39e3700 1 -- 172.21.15.146:0/1695567167 wait complete. 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T16:59:58.053 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T16:59:58.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.054 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-08-15T16:59:58.061 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T16:59:58.062 INFO:teuthology.orchestra.run.smithi146.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T16:59:58.063 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 02:42:1c:9c:24:e8 brd ff:ff:ff:ff:ff:ff 2024-08-15T16:59:58.063 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T16:59:58.063 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T16:59:58.063 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T16:59:58.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.063 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T16:59:58.063 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T16:59:58.063 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add name ceph-brx type bridge 2024-08-15T16:59:58.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr flush dev ceph-brx 2024-08-15T16:59:58.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set ceph-brx up 2024-08-15T16:59:58.064 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T16:59:58.064 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T16:59:58.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.145 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T16:59:58.154 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-08-15T16:59:58.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.156 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T16:59:58.180 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T16:59:58.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T16:59:58.181 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T16:59:58.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.219 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-08-15T16:59:58.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.223 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-08-15T16:59:58.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.274 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T16:59:58.274 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T16:59:58.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T16:59:58.274 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-15T16:59:58.274 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T16:59:58.306 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-15T16:59:58.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-15T16:59:58.307 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T16:59:58.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T16:59:58.379 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T16:59:58.380 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T16:59:58.380 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-08-15T16:59:58.380 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T16:59:58.380 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T16:59:58.406 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T16:59:58.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T16:59:58.406 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T16:59:58.414 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T16:59:58.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T16:59:58.414 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T16:59:58.465 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T16:59:58.484 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T16:59:58.484 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T16:59:58.499 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T16:59:58.499 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T16:59:58.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T16:59:58.500 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T16:59:58.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T16:59:58.505 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T16:59:58.549 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T16:59:58.565 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T16:59:58.565 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T16:59:58.576 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T16:59:58.578 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T16:59:58.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T16:59:58.579 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T16:59:58.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:2024-08-15T16:59:58.621+0000 7f231b1a4380 -1 init, newargv = 0x5579e7a319b0 newargc=15 2024-08-15T16:59:58.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[31470]: starting ceph client 2024-08-15T16:59:58.641 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[31470]: starting fuse 2024-08-15T16:59:59.685 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T16:59:59.704 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T16:59:59.704 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T16:59:59.763 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T16:59:59.763 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T16:59:59.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T16:59:59.764 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T16:59:59.771 INFO:teuthology.orchestra.run.smithi146.stdout:51 2024-08-15T16:59:59.771 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [51] 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-08-15T16:59:59.772 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.0")) 2024-08-15T16:59:59.773 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-08-15T16:59:59.855 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.0.31470.asok 2024-08-15T17:00:00.038 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31470.asok 2024-08-15T17:00:00.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:00.038 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31470.asok status 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "0", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "31470", 2024-08-15T17:00:00.132 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4620, 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4620 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1181167075 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:00.133 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 1181167075 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4620 192.168.0.1:0/1181167075", 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/1181167075", 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 6, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 21, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-08-15T17:00:00.134 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:00:00.142 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:00:00.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.233+0000 7f11d1adb700 1 Processor -- start 2024-08-15T17:00:00.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.233+0000 7f11d1adb700 1 -- start start 2024-08-15T17:00:00.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f11cc159170 0x7f11cc164060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:00.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0aa270 con 0x7f11cc068c10 2024-08-15T17:00:00.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11cc0aa550 con 0x7f11cc068840 2024-08-15T17:00:00.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11cc069090 con 0x7f11cc159170 2024-08-15T17:00:00.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cbfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f11cc068c10 0x7f11cc158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:42004/0 (socket says 172.21.15.146:42004) 2024-08-15T17:00:00.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cbfff700 1 -- 172.21.15.146:0/3539692158 learned_addr learned my addr 172.21.15.146:0/3539692158 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:00.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d12da700 1 --2- 172.21.15.146:0/3539692158 >> v2:172.21.15.146:3300/0 conn(0x7f11cc159170 0x7f11cc164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:00.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617022616 0 0) 0x7f11cc0aa270 con 0x7f11cc068c10 2024-08-15T17:00:00.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11bc003570 con 0x7f11cc068c10 2024-08-15T17:00:00.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d12da700 1 -- 172.21.15.146:0/3539692158 >> v1:172.21.15.161:6789/0 conn(0x7f11cc068840 legacy=0x7f11cc0a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179473285 0 0) 0x7f11bc003570 con 0x7f11cc068c10 2024-08-15T17:00:00.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d12da700 1 -- 172.21.15.146:0/3539692158 >> v1:172.21.15.146:6789/0 conn(0x7f11cc068c10 legacy=0x7f11cc158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d12da700 1 -- 172.21.15.146:0/3539692158 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0aa830 con 0x7f11cc159170 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d12da700 1 --2- 172.21.15.146:0/3539692158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc164060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009e90 tx=0x7f11c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a468ab2915dbb4a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400e070 con 0x7f11cc159170 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400b920 con 0x7f11cc159170 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11cb7fe700 1 -- 172.21.15.146:0/3539692158 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0001470 con 0x7f11cc159170 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- 172.21.15.146:0/3539692158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 msgr2=0x7f11cc164060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3539692158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc164060 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009e90 tx=0x7f11c4009780 comp rx=0 tx=0).stop 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- 172.21.15.146:0/3539692158 shutdown_connections 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3539692158 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc164060 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- 172.21.15.146:0/3539692158 >> 172.21.15.146:0/3539692158 conn(0x7f11cc06d800 msgr2=0x7f11cc06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- 172.21.15.146:0/3539692158 shutdown_connections 2024-08-15T17:00:00.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.237+0000 7f11d1adb700 1 -- 172.21.15.146:0/3539692158 wait complete. 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 Processor -- start 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- start start 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11cc068840 0x7f11cc144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11cc068c10 0x7f11cc145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0c31a0 con 0x7f11cc159170 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0c2590 con 0x7f11cc068840 2024-08-15T17:00:00.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11cc0c2870 con 0x7f11cc068c10 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56862/0 (socket says 172.21.15.146:56862) 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 -- 172.21.15.146:0/3482495018 learned_addr learned my addr 172.21.15.146:0/3482495018 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11cbfff700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11cc068c10 0x7f11cc145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d0ad9700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11cc068840 0x7f11cc144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 -- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11cc068c10 msgr2=0x7f11cc145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11cc068c10 0x7f11cc145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 -- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11cc068840 msgr2=0x7f11cc144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11cc068840 0x7f11cc144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 -- 172.21.15.146:0/3482495018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4009a20 con 0x7f11cc159170 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d12da700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11cc069650 tx=0x7f11c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4012c20 con 0x7f11cc159170 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c4012d80 con 0x7f11cc159170 2024-08-15T17:00:00.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40266b0 con 0x7f11cc159170 2024-08-15T17:00:00.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc0c2b50 con 0x7f11cc159170 2024-08-15T17:00:00.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.241+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11cc144c60 con 0x7f11cc159170 2024-08-15T17:00:00.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11cc0c3310 con 0x7f11cc159170 2024-08-15T17:00:00.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f11c4026ae0 con 0x7f11cc159170 2024-08-15T17:00:00.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11c8ff9700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 0x7f11b0062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:00.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11c407ecf0 con 0x7f11cc159170 2024-08-15T17:00:00.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11d0ad9700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 0x7f11b0062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:00.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.245+0000 7f11d0ad9700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 0x7f11b0062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11b8003b80 tx=0x7f11b8007e40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:00.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.249+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f11c4083070 con 0x7f11cc159170 2024-08-15T17:00:00.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.449+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11cc0638f0 con 0x7f11cc159170 2024-08-15T17:00:00.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.449+0000 7f11c8ff9700 1 -- 172.21.15.146:0/3482495018 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f11c404f140 con 0x7f11cc159170 2024-08-15T17:00:00.461 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:00:00.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 msgr2=0x7f11b0062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 0x7f11b0062d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f11b8003b80 tx=0x7f11b8007e40 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 msgr2=0x7f11cc0c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11cc069650 tx=0x7f11c4017000 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 shutdown_connections 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f11b0060890 0x7f11b0062d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f11cc159170 0x7f11cc0c1840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11cc068c10 0x7f11cc145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 --2- 172.21.15.146:0/3482495018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11cc068840 0x7f11cc144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 >> 172.21.15.146:0/3482495018 conn(0x7f11cc06d800 msgr2=0x7f11cc0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 shutdown_connections 2024-08-15T17:00:00.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:00.453+0000 7f11d1adb700 1 -- 172.21.15.146:0/3482495018 wait complete. 2024-08-15T17:00:00.474 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:00:00.474 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:00:00.474 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:00:00.475 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:00:00.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.475 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-08-15T17:00:00.483 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:00.483 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:00.484 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:00.484 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.484 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:00.484 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.485 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42585sec preferred_lft 42585sec 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 02:42:1c:9c:24:e8 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.486 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether e2:7c:0c:39:5d:35 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::98d9:a9ff:feb5:982a/64 scope link 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether e2:7c:0c:39:5d:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e07c:cff:fe39:5d35/64 scope link 2024-08-15T17:00:00.487 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.488 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:00:00.502 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-08-15T17:00:00.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.504 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-08-15T17:00:00.555 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-08-15T17:00:00.555 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:00:00.555 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:00:00.555 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:00:00.556 INFO:teuthology.orchestra.run.smithi146.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:00:00.556 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:00:00.556 INFO:teuthology.orchestra.run.smithi146.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:00:00.556 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:00:00.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:00:00.556 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T17:00:00.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.590 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-08-15T17:00:00.637 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:00:00.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.638 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list-id 2024-08-15T17:00:00.646 INFO:teuthology.orchestra.run.smithi146.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-08-15T17:00:00.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.646 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T17:00:00.646 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T17:00:00.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:00:00.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-08-15T17:00:00.647 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T17:00:00.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.673 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 5a:46:54:ea:84:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.688 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::5846:54ff:feea:84e8/64 scope link 2024-08-15T17:00:00.689 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.690 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 5a:46:54:ea:84:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:00:00.747 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:00:00.748 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.748 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::5846:54ff:feea:84e8/64 scope link 2024-08-15T17:00:00.748 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:00.749 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-08-15T17:00:00.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.749 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T17:00:00.749 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T17:00:00.749 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-08-15T17:00:00.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:00:00.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-08-15T17:00:00.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-08-15T17:00:00.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-08-15T17:00:00.750 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T17:00:00.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:00.803 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T17:00:00.803 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T17:00:00.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.1 up 2024-08-15T17:00:00.804 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:00:00.804 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T17:00:00.828 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-08-15T17:00:00.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:00.829 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:00.878 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-08-15T17:00:00.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:00.878 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:00.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:00.902 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:00.903 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:00.958 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:00.959 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:00.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:00.960 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:00.966 INFO:teuthology.orchestra.run.smithi146.stdout:51 2024-08-15T17:00:00.966 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [51] 2024-08-15T17:00:00.966 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-08-15T17:00:00.969 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:00.985 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:00.985 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:01.047 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:01.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:01.048 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:01.054 INFO:teuthology.orchestra.run.smithi146.stdout:51 2024-08-15T17:00:01.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi146.stderr:2024-08-15T17:00:01.045+0000 7f559def9380 -1 init, newargv = 0x55af4ea07380 newargc=15 2024-08-15T17:00:01.056 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi146.stderr:ceph-fuse[31608]: starting ceph client 2024-08-15T17:00:01.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi146.stderr:ceph-fuse[31608]: starting fuse 2024-08-15T17:00:02.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:02.174 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:02.174 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:02.185 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:02.186 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:02.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:02.186 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:02.233 INFO:teuthology.orchestra.run.smithi146.stdout:51 2024-08-15T17:00:02.234 INFO:teuthology.orchestra.run.smithi146.stdout:53 2024-08-15T17:00:02.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [51, 53] 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-08-15T17:00:02.234 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-08-15T17:00:02.235 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-08-15T17:00:02.236 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-08-15T17:00:02.236 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:00:02.236 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:02.236 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.2")) 2024-08-15T17:00:02.236 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-08-15T17:00:02.318 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.2.31608.asok 2024-08-15T17:00:02.503 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31608.asok 2024-08-15T17:00:02.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:02.504 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31608.asok status 2024-08-15T17:00:02.593 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:00:02.593 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-08-15T17:00:02.593 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:02.593 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "2", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "31608", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4635, 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-08-15T17:00:02.594 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 4635 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 851409818 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 851409818 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:02.595 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.4635 192.168.0.2:0/851409818", 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.2:0/851409818", 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 6, 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 21, 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-08-15T17:00:02.596 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:00:02.602 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:00:02.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 Processor -- start 2024-08-15T17:00:02.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 -- start start 2024-08-15T17:00:02.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f95780b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.712 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9578141a00 con 0x7f957806e620 2024-08-15T17:00:02.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9578140b10 con 0x7f95780b0d20 2024-08-15T17:00:02.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9578140df0 con 0x7f9578142ea0 2024-08-15T17:00:02.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9575d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f957806e620 0x7f95780b10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:42020/0 (socket says 172.21.15.146:42020) 2024-08-15T17:00:02.713 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9575d9b700 1 -- 172.21.15.146:0/1261682777 learned_addr learned my addr 172.21.15.146:0/1261682777 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:02.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f95780b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474659373 0 0) 0x7f9578141a00 con 0x7f957806e620 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9564003570 con 0x7f957806e620 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129700467 0 0) 0x7f9578140b10 con 0x7f95780b0d20 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9578141a00 con 0x7f95780b0d20 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911711049 0 0) 0x7f9564003570 con 0x7f957806e620 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 msgr2=0x7f95780b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f95780b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 >> v1:172.21.15.161:6789/0 conn(0x7f95780b0d20 legacy=0x7f957806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95781410d0 con 0x7f957806e620 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f95700017f0 con 0x7f957806e620 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 >> v1:172.21.15.146:6789/0 conn(0x7f957806e620 legacy=0x7f95780b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9564003c20 0x7f9564004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f9564004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9564004d50 0x7f95640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9578140df0 con 0x7f9564004d50 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956400bb70 con 0x7f9564003c20 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956400bd30 con 0x7f9578142ea0 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9564004d50 0x7f95640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f957659c700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9564003c20 0x7f9564004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9575d9b700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f9564004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 -- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 msgr2=0x7f9564004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9578142ea0 0x7f9564004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 -- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9564003c20 msgr2=0x7f9564004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9564003c20 0x7f9564004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.701+0000 7f9576d9d700 1 -- 172.21.15.146:0/1261682777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564003570 con 0x7f9564004d50 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f9576d9d700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9564004d50 0x7f95640051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9560009c00 tx=0x7f95600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d29fa6a09828f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000e710 con 0x7f9564004d50 2024-08-15T17:00:02.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000e870 con 0x7f9564004d50 2024-08-15T17:00:02.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957559a700 1 -- 172.21.15.146:0/1261682777 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000ec30 con 0x7f9564004d50 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- 172.21.15.146:0/1261682777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9564004d50 msgr2=0x7f95640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 --2- 172.21.15.146:0/1261682777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9564004d50 0x7f95640051c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9560009c00 tx=0x7f95600091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f9576d9d700 1 -- 172.21.15.146:0/1261682777 reap_dead start 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- 172.21.15.146:0/1261682777 shutdown_connections 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- 172.21.15.146:0/1261682777 >> 172.21.15.146:0/1261682777 conn(0x7f95780678d0 msgr2=0x7f9578067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- 172.21.15.146:0/1261682777 shutdown_connections 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- 172.21.15.146:0/1261682777 wait complete. 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 Processor -- start 2024-08-15T17:00:02.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.705+0000 7f957759e700 1 -- start start 2024-08-15T17:00:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780a9f50 0x7f95780a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95780aa320 0x7f957806cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957816eb80 con 0x7f957806d440 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957806d8b0 con 0x7f95780a9f50 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957816e8a0 con 0x7f95780aa320 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56876/0 (socket says 172.21.15.146:56876) 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 -- 172.21.15.146:0/3627910186 learned_addr learned my addr 172.21.15.146:0/3627910186 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9575d9b700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95780aa320 0x7f957806cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957659c700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780a9f50 0x7f95780a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 -- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95780aa320 msgr2=0x7f957806cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95780aa320 0x7f957806cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 -- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780a9f50 msgr2=0x7f95780a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780a9f50 0x7f95780a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 -- 172.21.15.146:0/3627910186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780a5580 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f9576d9d700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f956400bb40 tx=0x7f956001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560022070 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956000ebf0 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95780a5860 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956000e710 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957816e140 con 0x7f957806d440 2024-08-15T17:00:02.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.709+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957816eb80 con 0x7f957806d440 2024-08-15T17:00:02.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.713+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f956001c950 con 0x7f957806d440 2024-08-15T17:00:02.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.713+0000 7f956e7fc700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 0x7f9558062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:02.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.713+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9560000e40 con 0x7f957806d440 2024-08-15T17:00:02.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.713+0000 7f957659c700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 0x7f9558062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:02.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.713+0000 7f957659c700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 0x7f9558062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9568007e40 tx=0x7f9568007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:02.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.721+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f956004e690 con 0x7f957806d440 2024-08-15T17:00:02.936 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.925+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f95780643a0 con 0x7f957806d440 2024-08-15T17:00:02.937 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.925+0000 7f956e7fc700 1 -- 172.21.15.146:0/3627910186 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9560051cb0 con 0x7f957806d440 2024-08-15T17:00:02.937 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:00:02.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 msgr2=0x7f9558062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 0x7f9558062d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9568007e40 tx=0x7f9568007d30 comp rx=0 tx=0).stop 2024-08-15T17:00:02.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 msgr2=0x7f957816dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f956400bb40 tx=0x7f956001a000 comp rx=0 tx=0).stop 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 shutdown_connections 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9558060890 0x7f9558062d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f957806d440 0x7f957816dc00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95780aa320 0x7f957806cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 --2- 172.21.15.146:0/3627910186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95780a9f50 0x7f95780a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 >> 172.21.15.146:0/3627910186 conn(0x7f95780678d0 msgr2=0x7f95780caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 shutdown_connections 2024-08-15T17:00:02.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:02.929+0000 7f957759e700 1 -- 172.21.15.146:0/3627910186 wait complete. 2024-08-15T17:00:02.951 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:00:02.952 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:00:02.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:02.952 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-08-15T17:00:02.960 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:02.960 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:02.960 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:02.960 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-08-15T17:00:02.961 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:7f:7e:3a:c4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:02.962 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-15T17:00:02.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:02.962 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:02.962 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:02.962 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2024-08-15T17:00:02.963 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2024-08-15T17:00:02.963 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2024-08-15T17:00:02.963 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-15T17:00:02.963 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:03.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.016 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:00:03.027 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-08-15T17:00:03.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.029 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-08-15T17:00:03.053 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:00:03.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:00:03.055 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:03.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.096 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-08-15T17:00:03.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.104 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-08-15T17:00:03.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.109 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:03.109 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:03.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:00:03.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-08-15T17:00:03.109 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:03.137 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-08-15T17:00:03.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.137 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-08-15T17:00:03.138 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:03.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-08-15T17:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2024-08-15T17:00:03.242 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:03.268 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:00:03.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:03.268 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:03.274 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:00:03.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:03.274 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:03.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:03.344 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:03.344 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:03.360 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:03.361 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:03.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:03.362 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:03.368 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:00:03.368 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:00:03.412 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:03.430 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:03.430 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:03.439 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:03.439 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:03.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:03.440 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:03.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:2024-08-15T17:00:03.485+0000 7fc25e0f3380 -1 init, newargv = 0x55e8cfd1bd40 newargc=15 2024-08-15T17:00:03.501 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:ceph-fuse[29138]: starting ceph client 2024-08-15T17:00:03.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:ceph-fuse[29138]: starting fuse 2024-08-15T17:00:04.595 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:04.613 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:04.613 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:04.627 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:04.628 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:04.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:04.629 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:04.681 INFO:teuthology.orchestra.run.smithi161.stdout:52 2024-08-15T17:00:04.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> import glob 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> import re 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> def _find_admin_socket(client_name): 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> files = glob.glob(asok_path) 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:00:04.682 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> # Given a non-glob path, it better be there 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> if "*" not in asok_path: 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> assert(len(files) == 1) 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> return files[0] 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> for f in files: 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> contents = proc_f.read() 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> if mountpoint in contents: 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> return f 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:00:04.683 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:04.684 DEBUG:teuthology.orchestra.run.smithi161:> print(_find_admin_socket("client.1")) 2024-08-15T17:00:04.684 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-08-15T17:00:04.765 INFO:teuthology.orchestra.run.smithi161.stdout:/var/run/ceph/ceph-client.1.29138.asok 2024-08-15T17:00:04.951 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29138.asok 2024-08-15T17:00:04.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:04.951 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29138.asok status 2024-08-15T17:00:05.044 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-15T17:00:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": { 2024-08-15T17:00:05.044 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "entity_id": "1", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": "smithi161", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "pid": "29138", 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "root": "/" 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_count": 0, 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 4678, 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "inst": { 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "name": { 2024-08-15T17:00:05.045 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "client", 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "num": 4678 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 3678870871 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:05.046 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 3678870871 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "inst_str": "client.4678 192.168.0.1:0/3678870871", 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "addr_str": "192.168.0.1:0/3678870871", 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "inode_count": 1, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_epoch": 6, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch": 21, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklisted": false, 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs" 2024-08-15T17:00:05.047 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-15T17:00:05.053 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:00:05.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 Processor -- start 2024-08-15T17:00:05.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 -- start start 2024-08-15T17:00:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f83240cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 0x7f8324142850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240c9800 con 0x7f832415c9b0 2024-08-15T17:00:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83240cc4c0 con 0x7f83241423d0 2024-08-15T17:00:05.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8329951700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324142d90 con 0x7f832415c5e0 2024-08-15T17:00:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f83240cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f83237fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 0x7f8324142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f83227fc700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f832415c9b0 0x7f83240c8fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:47130/0 (socket says 172.21.15.146:47130) 2024-08-15T17:00:05.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f83227fc700 1 -- 172.21.15.146:0/3818528254 learned_addr learned my addr 172.21.15.146:0/3818528254 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 -- 172.21.15.146:0/3818528254 >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 msgr2=0x7f8324142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 --2- 172.21.15.146:0/3818528254 >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 0x7f8324142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8321ffb700 1 -- 172.21.15.146:0/3818528254 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375026348 0 0) 0x7f83240c9800 con 0x7f832415c9b0 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 -- 172.21.15.146:0/3818528254 >> v1:172.21.15.161:6789/0 conn(0x7f832415c9b0 legacy=0x7f83240c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 -- 172.21.15.146:0/3818528254 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240c9ae0 con 0x7f832415c5e0 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f83237fe700 1 --2- 172.21.15.146:0/3818528254 >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 0x7f8324142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:05.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.157+0000 7f8322ffd700 1 --2- 172.21.15.146:0/3818528254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f83240cbf80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f831c0098e0 tx=0x7f831c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131e5c94902715b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8321ffb700 1 -- 172.21.15.146:0/3818528254 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00b640 con 0x7f832415c5e0 2024-08-15T17:00:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8321ffb700 1 -- 172.21.15.146:0/3818528254 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00b7a0 con 0x7f832415c5e0 2024-08-15T17:00:05.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8321ffb700 1 -- 172.21.15.146:0/3818528254 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00bb40 con 0x7f832415c5e0 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- 172.21.15.146:0/3818528254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 msgr2=0x7f83240cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 --2- 172.21.15.146:0/3818528254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f83240cbf80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f831c0098e0 tx=0x7f831c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- 172.21.15.146:0/3818528254 shutdown_connections 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 --2- 172.21.15.146:0/3818528254 >> v2:172.21.15.161:3300/0 conn(0x7f83241423d0 0x7f8324142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 --2- 172.21.15.146:0/3818528254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f83240cbf80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- 172.21.15.146:0/3818528254 >> 172.21.15.146:0/3818528254 conn(0x7f83240caad0 msgr2=0x7f83240caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- 172.21.15.146:0/3818528254 shutdown_connections 2024-08-15T17:00:05.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- 172.21.15.146:0/3818528254 wait complete. 2024-08-15T17:00:05.174 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 Processor -- start 2024-08-15T17:00:05.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.161+0000 7f8329951700 1 -- start start 2024-08-15T17:00:05.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83241423d0 0x7f832416aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f832416c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832413c870 con 0x7f832415c9b0 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832413cb50 con 0x7f83241423d0 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832413ce30 con 0x7f832415c5e0 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:56880/0 (socket says 172.21.15.146:56880) 2024-08-15T17:00:05.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 -- 172.21.15.146:0/1400344863 learned_addr learned my addr 172.21.15.146:0/1400344863 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83227fc700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f832416c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8322ffd700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83241423d0 0x7f832416aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 -- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 msgr2=0x7f832416c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f832416c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 -- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83241423d0 msgr2=0x7f832416aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83241423d0 0x7f832416aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 -- 172.21.15.146:0/1400344863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8314009420 con 0x7f832415c9b0 2024-08-15T17:00:05.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83227fc700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f832416c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f83237fe700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea20 tx=0x7f83140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400ee60 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400c850 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f831c009420 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400cc10 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.165+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832413d3f0 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240645c0 con 0x7f832415c9b0 2024-08-15T17:00:05.180 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8314013470 con 0x7f832415c9b0 2024-08-15T17:00:05.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f831affd700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:05.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f831407cbb0 con 0x7f832415c9b0 2024-08-15T17:00:05.181 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f8322ffd700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:05.182 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.169+0000 7f8322ffd700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 0x7f8308062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f831c0091e0 tx=0x7f831c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:05.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.177+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f831404cf60 con 0x7f832415c9b0 2024-08-15T17:00:05.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.377+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83240cbd60 con 0x7f832415c9b0 2024-08-15T17:00:05.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.377+0000 7f831affd700 1 -- 172.21.15.146:0/1400344863 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8314050580 con 0x7f832415c9b0 2024-08-15T17:00:05.391 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:00:05.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 msgr2=0x7f8308062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 0x7f8308062d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f831c0091e0 tx=0x7f831c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:05.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 msgr2=0x7f83240c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f831400ea20 tx=0x7f83140091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 shutdown_connections 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8308060860 0x7f8308062d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f832415c9b0 0x7f83240c7830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f832415c5e0 0x7f832416c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 --2- 172.21.15.146:0/1400344863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83241423d0 0x7f832416aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 >> 172.21.15.146:0/1400344863 conn(0x7f83240caad0 msgr2=0x7f8324068570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.381+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 shutdown_connections 2024-08-15T17:00:05.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:05.385+0000 7f8329951700 1 -- 172.21.15.146:0/1400344863 wait complete. 2024-08-15T17:00:05.410 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:00:05.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:00:05.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-08-15T17:00:05.410 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-08-15T17:00:05.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:05.411 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:00:05.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:00:05.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:00:05.411 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:00:05.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.411 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:05.419 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.420 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:05.420 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.420 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:05.420 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.420 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-08-15T17:00:05.421 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-08-15T17:00:05.422 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.422 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:00:05.422 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:7f:7e:3a:c4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.422 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:00:05.422 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 36:5a:c7:61:1d:25 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::4b6:5cff:fe2f:5508/64 scope link 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 36:5a:c7:61:1d:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::345a:c7ff:fe61:1d25/64 scope link 2024-08-15T17:00:05.423 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.424 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:00:05.439 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-08-15T17:00:05.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.441 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:00:05.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:00:05.497 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:05.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.537 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-08-15T17:00:05.544 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:00:05.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.545 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-08-15T17:00:05.593 INFO:teuthology.orchestra.run.smithi161.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-08-15T17:00:05.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.594 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:05.594 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:05.594 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:00:05.594 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-08-15T17:00:05.594 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:05.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.627 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:05.637 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 92:60:b1:66:76:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::9060:b1ff:fe66:762e/64 scope link 2024-08-15T17:00:05.638 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.639 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-15T17:00:05.699 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 92:60:b1:66:76:2e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::9060:b1ff:fe66:762e/64 scope link 2024-08-15T17:00:05.700 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:05.701 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-08-15T17:00:05.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-08-15T17:00:05.702 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:05.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:05.764 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:05.764 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:05.764 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.1 up 2024-08-15T17:00:05.764 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.1 master ceph-brx 2024-08-15T17:00:05.764 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:05.789 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-08-15T17:00:05.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:05.789 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:05.837 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-08-15T17:00:05.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:05.838 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:05.844 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:05.861 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:05.862 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:05.918 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:05.919 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:05.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:05.920 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:05.927 INFO:teuthology.orchestra.run.smithi161.stdout:52 2024-08-15T17:00:05.927 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-08-15T17:00:05.928 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-08-15T17:00:05.930 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:05.946 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:05.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:06.000 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:06.001 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:06.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:06.002 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:06.006 INFO:teuthology.orchestra.run.smithi161.stdout:52 2024-08-15T17:00:06.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi161.stderr:2024-08-15T17:00:06.001+0000 7fbe956e6380 -1 init, newargv = 0x5567685e74c0 newargc=15 2024-08-15T17:00:06.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi161.stderr:ceph-fuse[29251]: starting ceph client 2024-08-15T17:00:06.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi161.stderr:ceph-fuse[29251]: starting fuse 2024-08-15T17:00:07.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:07.125 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:07.125 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:07.180 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:07.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.182 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:07.187 INFO:teuthology.orchestra.run.smithi161.stdout:52 2024-08-15T17:00:07.187 INFO:teuthology.orchestra.run.smithi161.stdout:53 2024-08-15T17:00:07.187 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> import glob 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> import re 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> def _find_admin_socket(client_name): 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> files = glob.glob(asok_path) 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> # Given a non-glob path, it better be there 2024-08-15T17:00:07.188 DEBUG:teuthology.orchestra.run.smithi161:> if "*" not in asok_path: 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> assert(len(files) == 1) 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> return files[0] 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> for f in files: 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> contents = proc_f.read() 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> if mountpoint in contents: 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> return f 2024-08-15T17:00:07.189 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:00:07.190 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:07.190 DEBUG:teuthology.orchestra.run.smithi161:> print(_find_admin_socket("client.3")) 2024-08-15T17:00:07.190 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-08-15T17:00:07.270 INFO:teuthology.orchestra.run.smithi161.stdout:/var/run/ceph/ceph-client.3.29251.asok 2024-08-15T17:00:07.456 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.29251.asok 2024-08-15T17:00:07.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.29251.asok status 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": { 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "entity_id": "3", 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": "smithi161", 2024-08-15T17:00:07.598 INFO:teuthology.orchestra.run.smithi161.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "pid": "29251", 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "root": "/" 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_count": 0, 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 4541, 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "inst": { 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "name": { 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "client", 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "num": 4541 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:07.599 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4015792860 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.2:0", 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4015792860 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "inst_str": "client.4541 192.168.0.2:0/4015792860", 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "addr_str": "192.168.0.2:0/4015792860", 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "inode_count": 1, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_epoch": 6, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch": 21, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:07.600 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklisted": false, 2024-08-15T17:00:07.601 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs" 2024-08-15T17:00:07.601 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-15T17:00:07.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.607 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-08-15T17:00:07.607 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:07.617 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-08-15T17:00:07.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:07.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.617 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:07.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.683 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-08-15T17:00:07.683 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:07.695 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-08-15T17:00:07.695 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:07.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:07.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.715 DEBUG:teuthology.orchestra.run.smithi161:> stat --file-system '--printf=%T 2024-08-15T17:00:07.715 DEBUG:teuthology.orchestra.run.smithi161:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:07.724 INFO:teuthology.orchestra.run.smithi161.stdout:fuseblk 2024-08-15T17:00:07.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:07.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.724 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:07.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.783 DEBUG:teuthology.orchestra.run.smithi161:> stat --file-system '--printf=%T 2024-08-15T17:00:07.784 DEBUG:teuthology.orchestra.run.smithi161:> ' -- /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:07.792 INFO:teuthology.orchestra.run.smithi161.stdout:fuseblk 2024-08-15T17:00:07.793 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:07.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:07.793 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:07.851 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-08-15T17:00:08.180 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:08.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.281+0000 7f2fe88c2700 1 Processor -- start 2024-08-15T17:00:08.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- start start 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe40b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40cbaa0 con 0x7f2fe40b12d0 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40b1700 con 0x7f2fe40cb560 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4068790 con 0x7f2fe4154900 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe259c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2fe40b12d0 0x7f2fe40cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39300/0 (socket says 172.21.15.146:39300) 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe259c700 1 -- 172.21.15.146:0/1927220374 learned_addr learned my addr 172.21.15.146:0/1927220374 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:08.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/1927220374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe40b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044421208 0 0) 0x7f2fe40cbaa0 con 0x7f2fe40b12d0 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd0003570 con 0x7f2fe40b12d0 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200896180 0 0) 0x7f2fe40b1700 con 0x7f2fe40cb560 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe40cbaa0 con 0x7f2fe40cb560 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3026431971 0 0) 0x7f2fd0003570 con 0x7f2fe40b12d0 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe2d9d700 1 -- 172.21.15.146:0/1927220374 >> v1:172.21.15.161:6789/0 conn(0x7f2fe40cb560 legacy=0x7f2fe4160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe2d9d700 1 -- 172.21.15.146:0/1927220374 >> v1:172.21.15.146:6789/0 conn(0x7f2fe40b12d0 legacy=0x7f2fe40cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe2d9d700 1 -- 172.21.15.146:0/1927220374 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4157630 con 0x7f2fe4154900 2024-08-15T17:00:08.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/1927220374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe40b0d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009910 tx=0x7f2fcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a46b787e85c53fa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40017f0 con 0x7f2fe4154900 2024-08-15T17:00:08.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc011420 con 0x7f2fe4154900 2024-08-15T17:00:08.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe1d9b700 1 -- 172.21.15.146:0/1927220374 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0117c0 con 0x7f2fe4154900 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- 172.21.15.146:0/1927220374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 msgr2=0x7f2fe40b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/1927220374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe40b0d90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009910 tx=0x7f2fcc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- 172.21.15.146:0/1927220374 shutdown_connections 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/1927220374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe40b0d90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- 172.21.15.146:0/1927220374 >> 172.21.15.146:0/1927220374 conn(0x7f2fe40678d0 msgr2=0x7f2fe4067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- 172.21.15.146:0/1927220374 shutdown_connections 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.285+0000 7f2fe88c2700 1 -- 172.21.15.146:0/1927220374 wait complete. 2024-08-15T17:00:08.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 Processor -- start 2024-08-15T17:00:08.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 -- start start 2024-08-15T17:00:08.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 0x7f2fe406a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe416abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe416a710 con 0x7f2fe40cb560 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40aabe0 con 0x7f2fe40b12d0 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe88c2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40aaec0 con 0x7f2fe4154900 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55236/0 (socket says 172.21.15.146:55236) 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe259c700 1 -- 172.21.15.146:0/747034799 learned_addr learned my addr 172.21.15.146:0/747034799 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 0x7f2fe406a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.289+0000 7f2fe359e700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe416abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 -- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 msgr2=0x7f2fe416abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe416abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 -- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 msgr2=0x7f2fe406a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 0x7f2fe406a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 -- 172.21.15.146:0/747034799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc009420 con 0x7f2fe40cb560 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 0x7f2fe406a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe259c700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400cbe0 tx=0x7f2fd4007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004dc0 con 0x7f2fe40cb560 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe40ab1a0 con 0x7f2fe40cb560 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400a6a0 con 0x7f2fe40cb560 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe4152fc0 con 0x7f2fe40cb560 2024-08-15T17:00:08.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400aa20 con 0x7f2fe40cb560 2024-08-15T17:00:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe40643a0 con 0x7f2fe40cb560 2024-08-15T17:00:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2fd4007150 con 0x7f2fe40cb560 2024-08-15T17:00:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fd407c6b0 con 0x7f2fe40cb560 2024-08-15T17:00:08.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.293+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.297+0000 7f2fe2d9d700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 0x7f2fc4062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0030c0 tx=0x7f2fcc016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.301+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2fd404cb00 con 0x7f2fe40cb560 2024-08-15T17:00:08.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.465+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fe406f5f0 con 0x7f2fe40cb560 2024-08-15T17:00:08.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.465+0000 7f2fdaffd700 1 -- 172.21.15.146:0/747034799 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2fd4018020 con 0x7f2fe40cb560 2024-08-15T17:00:08.478 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:08.478 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 msgr2=0x7f2fc4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 0x7f2fc4062d10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0030c0 tx=0x7f2fcc016000 comp rx=0 tx=0).stop 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 msgr2=0x7f2fe40a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400cbe0 tx=0x7f2fd4007d50 comp rx=0 tx=0).stop 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 shutdown_connections 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fc4060860 0x7f2fc4062d10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2fe4154900 0x7f2fe416abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2fe40cb560 0x7f2fe40a64f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 --2- 172.21.15.146:0/747034799 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fe40b12d0 0x7f2fe406a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 >> 172.21.15.146:0/747034799 conn(0x7f2fe40678d0 msgr2=0x7f2fe406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:08.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 shutdown_connections 2024-08-15T17:00:08.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.469+0000 7f2fe88c2700 1 -- 172.21.15.146:0/747034799 wait complete. 2024-08-15T17:00:08.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:08.492 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:08.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 Processor -- start 2024-08-15T17:00:08.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 -- start start 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1640067030 0x7f1640067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f16400cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f16400cb5c0 0x7f1640158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400cf670 con 0x7f1640067030 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16400cea60 con 0x7f16400cb5c0 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.593+0000 7f1646e06700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16400ced40 con 0x7f16400cfb00 2024-08-15T17:00:08.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1640067030 0x7f1640067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f1640067030 0x7f1640067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55246/0 (socket says 172.21.15.146:55246) 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645603700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f16400cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 -- 172.21.15.146:0/683158030 learned_addr learned my addr 172.21.15.146:0/683158030 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646605700 1 --2- 172.21.15.146:0/683158030 >> v2:172.21.15.161:3300/0 conn(0x7f16400cb5c0 0x7f1640158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 -- 172.21.15.146:0/683158030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 msgr2=0x7f16400cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f16400cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 -- 172.21.15.146:0/683158030 >> v2:172.21.15.161:3300/0 conn(0x7f16400cb5c0 msgr2=0x7f1640158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 --2- 172.21.15.146:0/683158030 >> v2:172.21.15.161:3300/0 conn(0x7f16400cb5c0 0x7f1640158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 -- 172.21.15.146:0/683158030 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16400cf020 con 0x7f1640067030 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645603700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f16400cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1645e04700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f1640067400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164006fb70 tx=0x7f1630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3bc3c41c7485fe server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1644e02700 1 -- 172.21.15.146:0/683158030 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000b640 con 0x7f1640067030 2024-08-15T17:00:08.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1644e02700 1 -- 172.21.15.146:0/683158030 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163000b7a0 con 0x7f1640067030 2024-08-15T17:00:08.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1644e02700 1 -- 172.21.15.146:0/683158030 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000bb60 con 0x7f1640067030 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- 172.21.15.146:0/683158030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 msgr2=0x7f1640067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f1640067400 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f164006fb70 tx=0x7f1630009250 comp rx=0 tx=0).stop 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- 172.21.15.146:0/683158030 shutdown_connections 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 --2- 172.21.15.146:0/683158030 >> v2:172.21.15.161:3300/0 conn(0x7f16400cb5c0 0x7f1640158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f16400cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 --2- 172.21.15.146:0/683158030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f1640067400 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- 172.21.15.146:0/683158030 >> 172.21.15.146:0/683158030 conn(0x7f16400d0b50 msgr2=0x7f16400ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- 172.21.15.146:0/683158030 shutdown_connections 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- 172.21.15.146:0/683158030 wait complete. 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 Processor -- start 2024-08-15T17:00:08.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.597+0000 7f1646e06700 1 -- start start 2024-08-15T17:00:08.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16400cb5c0 0x7f1640151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f1640158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16400c9940 con 0x7f1640067030 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164015abc0 con 0x7f16400cb5c0 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1640159600 con 0x7f16400cfb00 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55248/0 (socket says 172.21.15.146:55248) 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 -- 172.21.15.146:0/3366881300 learned_addr learned my addr 172.21.15.146:0/3366881300 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646605700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f1640158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645603700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16400cb5c0 0x7f1640151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 -- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 msgr2=0x7f1640158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f1640158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 -- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16400cb5c0 msgr2=0x7f1640151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16400cb5c0 0x7f1640151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 -- 172.21.15.146:0/3366881300 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c009420 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1645e04700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1640069520 tx=0x7f1630017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163000b910 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163000ba70 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16300094f0 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630011630 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164015a5f0 con 0x7f1640067030 2024-08-15T17:00:08.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.601+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16400638f0 con 0x7f1640067030 2024-08-15T17:00:08.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.605+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f16300119a0 con 0x7f1640067030 2024-08-15T17:00:08.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.605+0000 7f1635ffb700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 0x7f1624062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.605+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f163007e2e0 con 0x7f1640067030 2024-08-15T17:00:08.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.605+0000 7f1645603700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 0x7f1624062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.605+0000 7f1645603700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 0x7f1624062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f16400cb170 tx=0x7f163c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.613+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f163004e690 con 0x7f1640067030 2024-08-15T17:00:08.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f164015a970 con 0x7f1640067030 2024-08-15T17:00:08.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1635ffb700 1 -- 172.21.15.146:0/3366881300 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1630051cb0 con 0x7f1640067030 2024-08-15T17:00:08.784 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 msgr2=0x7f1624062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 0x7f1624062d10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f16400cb170 tx=0x7f163c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 msgr2=0x7f16400c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1640069520 tx=0x7f1630017000 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 shutdown_connections 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f1624060860 0x7f1624062d10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16400cfb00 0x7f1640158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16400cb5c0 0x7f1640151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 --2- 172.21.15.146:0/3366881300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1640067030 0x7f16400c9400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 >> 172.21.15.146:0/3366881300 conn(0x7f16400d0b50 msgr2=0x7f16400cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:08.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 shutdown_connections 2024-08-15T17:00:08.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.773+0000 7f1646e06700 1 -- 172.21.15.146:0/3366881300 wait complete. 2024-08-15T17:00:08.799 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:08.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.893+0000 7fd9ebfff700 1 Processor -- start 2024-08-15T17:00:08.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.893+0000 7fd9ebfff700 1 -- start start 2024-08-15T17:00:08.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec06e010 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec06efa0 con 0x7fd9ec1574f0 2024-08-15T17:00:08.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec157fb0 con 0x7fd9ec1514d0 2024-08-15T17:00:08.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ea7fc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eb7fe700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fd9ec0b0d90 0x7fd9ec0c96d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:39250/0 (socket says 172.21.15.146:39250) 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ea7fc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:59670/0 (socket says 172.21.15.146:59670) 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ea7fc700 1 -- 172.21.15.146:0/818677318 learned_addr learned my addr 172.21.15.146:0/818677318 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 -- 172.21.15.146:0/818677318 >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 msgr2=0x7fd9ec157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 --2- 172.21.15.146:0/818677318 >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 -- 172.21.15.146:0/818677318 >> v1:172.21.15.161:6789/0 conn(0x7fd9ec0b0d90 legacy=0x7fd9ec0c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 -- 172.21.15.146:0/818677318 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec158290 con 0x7fd9ec1514d0 2024-08-15T17:00:08.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ea7fc700 1 --2- 172.21.15.146:0/818677318 >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:08.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9eaffd700 1 --2- 172.21.15.146:0/818677318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec06ea60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0098e0 tx=0x7fd9dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676416fb1ab2c7da server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9e9ffb700 1 -- 172.21.15.146:0/818677318 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00b640 con 0x7fd9ec1514d0 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9e9ffb700 1 -- 172.21.15.146:0/818677318 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00b7a0 con 0x7fd9ec1514d0 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9e9ffb700 1 -- 172.21.15.146:0/818677318 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00bb40 con 0x7fd9ec1514d0 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- 172.21.15.146:0/818677318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 msgr2=0x7fd9ec06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/818677318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec06ea60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0098e0 tx=0x7fd9dc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- 172.21.15.146:0/818677318 shutdown_connections 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/818677318 >> v2:172.21.15.161:3300/0 conn(0x7fd9ec1574f0 0x7fd9ec157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/818677318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec06ea60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- 172.21.15.146:0/818677318 >> 172.21.15.146:0/818677318 conn(0x7fd9ec0678a0 msgr2=0x7fd9ec067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- 172.21.15.146:0/818677318 shutdown_connections 2024-08-15T17:00:08.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.897+0000 7fd9ebfff700 1 -- 172.21.15.146:0/818677318 wait complete. 2024-08-15T17:00:08.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 Processor -- start 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 -- start start 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9ec1574f0 0x7fd9ec1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec154cc0 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec153e50 con 0x7fd9ec1574f0 2024-08-15T17:00:08.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ebfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec139780 con 0x7fd9ec1514d0 2024-08-15T17:00:08.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ea7fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55262/0 (socket says 172.21.15.146:55262) 2024-08-15T17:00:08.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eb7fe700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9ec1574f0 0x7fd9ec1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 -- 172.21.15.146:0/2855073358 learned_addr learned my addr 172.21.15.146:0/2855073358 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 -- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 msgr2=0x7fd9ec155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 -- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9ec1574f0 msgr2=0x7fd9ec1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9ec1574f0 0x7fd9ec1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 -- 172.21.15.146:0/2855073358 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40094f0 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9ea7fc700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9eaffd700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec1580f0 tx=0x7fd9dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc01bc40 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc01bde0 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.901+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc011670 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc009420 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec154390 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec06f2d0 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd9dc0119e0 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9e2ffd700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 0x7fd9d0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd9dc07e160 con 0x7fd9ec0b0d90 2024-08-15T17:00:08.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.905+0000 7fd9ea7fc700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 0x7fd9d0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:08.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.909+0000 7fd9ea7fc700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 0x7fd9d0062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec14f2d0 tx=0x7fd9d4009250 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:08.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:08.913+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd9dc04e5b0 con 0x7fd9ec0b0d90 2024-08-15T17:00:09.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.073+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9ec064330 con 0x7fd9ec0b0d90 2024-08-15T17:00:09.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.073+0000 7fd9e2ffd700 1 -- 172.21.15.146:0/2855073358 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9dc051bd0 con 0x7fd9ec0b0d90 2024-08-15T17:00:09.087 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:09.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 msgr2=0x7fd9d0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 0x7fd9d0062d10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec14f2d0 tx=0x7fd9d4009250 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 msgr2=0x7fd9ec139240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec1580f0 tx=0x7fd9dc017000 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 shutdown_connections 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd9d0060860 0x7fd9d0062d10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd9ec1574f0 0x7fd9ec1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd9ec1514d0 0x7fd9ec155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 --2- 172.21.15.146:0/2855073358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd9ec0b0d90 0x7fd9ec139240 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 >> 172.21.15.146:0/2855073358 conn(0x7fd9ec0678a0 msgr2=0x7fd9ec155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 shutdown_connections 2024-08-15T17:00:09.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.077+0000 7fd9ebfff700 1 -- 172.21.15.146:0/2855073358 wait complete. 2024-08-15T17:00:09.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:09.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.197+0000 7f0f508ea700 1 Processor -- start 2024-08-15T17:00:09.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.197+0000 7f0f508ea700 1 -- start start 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f4c067510 0x7f0f4c0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0f4c159480 0x7f0f4c0cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0ce210 con 0x7f0f4c1590b0 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c0cba00 con 0x7f0f4c067510 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c0cdc90 con 0x7f0f4c159480 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0f4c067510 0x7f0f4c0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4b7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0f4c1590b0 0x7f0f4c066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39310/0 (socket says 172.21.15.146:39310) 2024-08-15T17:00:09.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4b7fe700 1 -- 172.21.15.146:0/2132864900 learned_addr learned my addr 172.21.15.146:0/2132864900 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4bfff700 1 --2- 172.21.15.146:0/2132864900 >> v2:172.21.15.161:3300/0 conn(0x7f0f4c159480 0x7f0f4c0cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 -- 172.21.15.146:0/2132864900 >> v2:172.21.15.161:3300/0 conn(0x7f0f4c159480 msgr2=0x7f0f4c0cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 --2- 172.21.15.146:0/2132864900 >> v2:172.21.15.161:3300/0 conn(0x7f0f4c159480 0x7f0f4c0cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 -- 172.21.15.146:0/2132864900 >> v1:172.21.15.146:6789/0 conn(0x7f0f4c1590b0 legacy=0x7f0f4c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4a7fc700 1 -- 172.21.15.146:0/2132864900 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317131323 0 0) 0x7f0f4c0ce210 con 0x7f0f4c1590b0 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 -- 172.21.15.146:0/2132864900 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c0cb440 con 0x7f0f4c067510 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4affd700 1 --2- 172.21.15.146:0/2132864900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c067510 0x7f0f4c0ce6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c071060 tx=0x7f0f3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9799c15680638d8d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4a7fc700 1 -- 172.21.15.146:0/2132864900 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00b6b0 con 0x7f0f4c067510 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4a7fc700 1 -- 172.21.15.146:0/2132864900 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c00b810 con 0x7f0f4c067510 2024-08-15T17:00:09.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f4a7fc700 1 -- 172.21.15.146:0/2132864900 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00bbb0 con 0x7f0f4c067510 2024-08-15T17:00:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- 172.21.15.146:0/2132864900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c067510 msgr2=0x7f0f4c0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 --2- 172.21.15.146:0/2132864900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c067510 0x7f0f4c0ce6a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c071060 tx=0x7f0f3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- 172.21.15.146:0/2132864900 shutdown_connections 2024-08-15T17:00:09.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 --2- 172.21.15.146:0/2132864900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c067510 0x7f0f4c0ce6a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 --2- 172.21.15.146:0/2132864900 >> v2:172.21.15.161:3300/0 conn(0x7f0f4c159480 0x7f0f4c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- 172.21.15.146:0/2132864900 >> 172.21.15.146:0/2132864900 conn(0x7f0f4c0d0b90 msgr2=0x7f0f4c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- 172.21.15.146:0/2132864900 shutdown_connections 2024-08-15T17:00:09.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.201+0000 7f0f508ea700 1 -- 172.21.15.146:0/2132864900 wait complete. 2024-08-15T17:00:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 Processor -- start 2024-08-15T17:00:09.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 -- start start 2024-08-15T17:00:09.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f4c067510 0x7f0f4c15eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f4c1590b0 0x7f0f4c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15ea30 con 0x7f0f4c159480 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15de20 con 0x7f0f4c067510 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f508ea700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c15e100 con 0x7f0f4c1590b0 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f4bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f4bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55282/0 (socket says 172.21.15.146:55282) 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f4bfff700 1 -- 172.21.15.146:0/3180097064 learned_addr learned my addr 172.21.15.146:0/3180097064 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f4b7fe700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f4c067510 0x7f0f4c15eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.205+0000 7f0f4affd700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f4c1590b0 0x7f0f4c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 -- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f4c1590b0 msgr2=0x7f0f4c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f4c1590b0 0x7f0f4c15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 -- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f4c067510 msgr2=0x7f0f4c15eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f4c067510 0x7f0f4c15eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 -- 172.21.15.146:0/3180097064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c009420 con 0x7f0f4c159480 2024-08-15T17:00:09.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f4bfff700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007f20 tx=0x7f0f44009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400eb90 con 0x7f0f4c159480 2024-08-15T17:00:09.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400ed10 con 0x7f0f4c159480 2024-08-15T17:00:09.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c15e3e0 con 0x7f0f4c159480 2024-08-15T17:00:09.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44014670 con 0x7f0f4c159480 2024-08-15T17:00:09.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c0c88c0 con 0x7f0f4c159480 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0f440149e0 con 0x7f0f4c159480 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.209+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c15eba0 con 0x7f0f4c159480 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.213+0000 7f0f337fe700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 0x7f0f38062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.213+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f4407d220 con 0x7f0f4c159480 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.213+0000 7f0f4b7fe700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 0x7f0f38062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.213+0000 7f0f4b7fe700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 0x7f0f38062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007fd0 tx=0x7f0f40007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.221+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0f4404d5d0 con 0x7f0f4c159480 2024-08-15T17:00:09.390 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.377+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f4c064330 con 0x7f0f4c159480 2024-08-15T17:00:09.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.377+0000 7f0f337fe700 1 -- 172.21.15.146:0/3180097064 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f44050bf0 con 0x7f0f4c159480 2024-08-15T17:00:09.391 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:09.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 msgr2=0x7f0f38062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 0x7f0f38062d10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f40007fd0 tx=0x7f0f40007d30 comp rx=0 tx=0).stop 2024-08-15T17:00:09.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 msgr2=0x7f0f4c0c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f44007f20 tx=0x7f0f44009860 comp rx=0 tx=0).stop 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 shutdown_connections 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0f38060860 0x7f0f38062d10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0f4c159480 0x7f0f4c0c8380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f4c1590b0 0x7f0f4c15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 --2- 172.21.15.146:0/3180097064 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f4c067510 0x7f0f4c15eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 >> 172.21.15.146:0/3180097064 conn(0x7f0f4c0d0b90 msgr2=0x7f0f4c0b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 shutdown_connections 2024-08-15T17:00:09.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.381+0000 7f0f508ea700 1 -- 172.21.15.146:0/3180097064 wait complete. 2024-08-15T17:00:09.409 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:09.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 Processor -- start 2024-08-15T17:00:09.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 -- start start 2024-08-15T17:00:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0480af4d0 0x7fa0481557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa048068b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa048069040 con 0x7fa0481553e0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa048155e30 con 0x7fa0480af4d0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04e412700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048156110 con 0x7fa0480686a0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0480af4d0 0x7fa0481557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa0480af4d0 0x7fa0481557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55294/0 (socket says 172.21.15.146:55294) 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 -- 172.21.15.146:0/2636434618 learned_addr learned my addr 172.21.15.146:0/2636434618 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04dc11700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa048068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 -- 172.21.15.146:0/2636434618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 msgr2=0x7fa048068b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa048068b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 -- 172.21.15.146:0/2636434618 >> v1:172.21.15.161:6789/0 conn(0x7fa0481553e0 legacy=0x7fa0480b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 -- 172.21.15.146:0/2636434618 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0481563f0 con 0x7fa0480af4d0 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04dc11700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa048068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:09.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa04cc0f700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0480af4d0 0x7fa0481557b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa044009f20 tx=0x7fa044009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d1626cbbe5ada6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.505+0000 7fa03bfff700 1 -- 172.21.15.146:0/2636434618 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400e070 con 0x7fa0480af4d0 2024-08-15T17:00:09.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa03bfff700 1 -- 172.21.15.146:0/2636434618 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04400ba80 con 0x7fa0480af4d0 2024-08-15T17:00:09.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa03bfff700 1 -- 172.21.15.146:0/2636434618 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440092e0 con 0x7fa0480af4d0 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- 172.21.15.146:0/2636434618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0480af4d0 msgr2=0x7fa0481557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0480af4d0 0x7fa0481557b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa044009f20 tx=0x7fa044009840 comp rx=0 tx=0).stop 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- 172.21.15.146:0/2636434618 shutdown_connections 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa048068b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 --2- 172.21.15.146:0/2636434618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0480af4d0 0x7fa0481557b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- 172.21.15.146:0/2636434618 >> 172.21.15.146:0/2636434618 conn(0x7fa04806d800 msgr2=0x7fa04806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- 172.21.15.146:0/2636434618 shutdown_connections 2024-08-15T17:00:09.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- 172.21.15.146:0/2636434618 wait complete. 2024-08-15T17:00:09.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 Processor -- start 2024-08-15T17:00:09.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.509+0000 7fa04e412700 1 -- start start 2024-08-15T17:00:09.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa0480cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0480af4d0 0x7fa0480aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048169e10 con 0x7fa0481553e0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04816a0f0 con 0x7fa0480af4d0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480a9a00 con 0x7fa0480686a0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04cc0f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0480af4d0 0x7fa0480aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04d410700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa0480cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55296/0 (socket says 172.21.15.146:55296) 2024-08-15T17:00:09.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 -- 172.21.15.146:0/3582799020 learned_addr learned my addr 172.21.15.146:0/3582799020 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 -- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 msgr2=0x7fa0480cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa0480cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 -- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0480af4d0 msgr2=0x7fa0480aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0480af4d0 0x7fa0480aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 -- 172.21.15.146:0/3582799020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044009ae0 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04dc11700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e870 tx=0x7fa040008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040013410 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040013570 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400138f0 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0480a9ce0 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.513+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0480ab100 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa040013ca0 con 0x7fa0481553e0 2024-08-15T17:00:09.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0480a9860 con 0x7fa0481553e0 2024-08-15T17:00:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa0397fa700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 0x7fa02c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa04007c820 con 0x7fa0481553e0 2024-08-15T17:00:09.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa04d410700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 0x7fa02c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.517+0000 7fa04d410700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 0x7fa02c062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0039e0 tx=0x7fa03c007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.521+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa0480a9860 con 0x7fa0481553e0 2024-08-15T17:00:09.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.685+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0480a9860 con 0x7fa0481553e0 2024-08-15T17:00:09.699 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.685+0000 7fa0397fa700 1 -- 172.21.15.146:0/3582799020 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa0480a9860 con 0x7fa0481553e0 2024-08-15T17:00:09.699 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:09.699 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 msgr2=0x7fa02c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 0x7fa02c062d10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0039e0 tx=0x7fa03c007ec0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 msgr2=0x7fa048169370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e870 tx=0x7fa040008040 comp rx=0 tx=0).stop 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 shutdown_connections 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa02c060860 0x7fa02c062d10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0481553e0 0x7fa048169370 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0480af4d0 0x7fa0480aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 --2- 172.21.15.146:0/3582799020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0480686a0 0x7fa0480cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 >> 172.21.15.146:0/3582799020 conn(0x7fa04806d800 msgr2=0x7fa04814fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 shutdown_connections 2024-08-15T17:00:09.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.689+0000 7fa04e412700 1 -- 172.21.15.146:0/3582799020 wait complete. 2024-08-15T17:00:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:09.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:09.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 Processor -- start 2024-08-15T17:00:09.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 -- start start 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 0x7fba30142a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 0x7fba300ae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba300b1180 con 0x7fba300c82e0 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300c9310 con 0x7fba300ae510 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.813+0000 7fba37003700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300aee80 con 0x7fba30142660 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36802700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fba300c82e0 0x7fba300c8760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39312/0 (socket says 172.21.15.146:39312) 2024-08-15T17:00:09.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36802700 1 -- 172.21.15.146:0/871369327 learned_addr learned my addr 172.21.15.146:0/871369327 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba35800700 1 --2- 172.21.15.146:0/871369327 >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 0x7fba300ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 --2- 172.21.15.146:0/871369327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 0x7fba30142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049625680 0 0) 0x7fba300b1180 con 0x7fba300c82e0 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba1c003570 con 0x7fba300c82e0 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 -- 172.21.15.146:0/871369327 >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 msgr2=0x7fba300ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 --2- 172.21.15.146:0/871369327 >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 0x7fba300ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 -- 172.21.15.146:0/871369327 >> v1:172.21.15.146:6789/0 conn(0x7fba300c82e0 legacy=0x7fba300c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4073267439 0 0) 0x7fba1c003570 con 0x7fba300c82e0 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 -- 172.21.15.146:0/871369327 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300b0890 con 0x7fba30142660 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba35800700 1 --2- 172.21.15.146:0/871369327 >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 0x7fba300ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:09.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba36001700 1 --2- 172.21.15.146:0/871369327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 0x7fba30142a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba28009e60 tx=0x7fba28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80453bb40192ad26 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2800e070 con 0x7fba30142660 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2800b7a0 con 0x7fba30142660 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba34fff700 1 -- 172.21.15.146:0/871369327 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20001470 con 0x7fba30142660 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 -- 172.21.15.146:0/871369327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 msgr2=0x7fba30142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 --2- 172.21.15.146:0/871369327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 0x7fba30142a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba28009e60 tx=0x7fba28009780 comp rx=0 tx=0).stop 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 -- 172.21.15.146:0/871369327 shutdown_connections 2024-08-15T17:00:09.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 --2- 172.21.15.146:0/871369327 >> v2:172.21.15.161:3300/0 conn(0x7fba300ae510 0x7fba300ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 --2- 172.21.15.146:0/871369327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba30142660 0x7fba30142a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 -- 172.21.15.146:0/871369327 >> 172.21.15.146:0/871369327 conn(0x7fba300c9830 msgr2=0x7fba300c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:09.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 -- 172.21.15.146:0/871369327 shutdown_connections 2024-08-15T17:00:09.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.817+0000 7fba37003700 1 -- 172.21.15.146:0/871369327 wait complete. 2024-08-15T17:00:09.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 Processor -- start 2024-08-15T17:00:09.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- start start 2024-08-15T17:00:09.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba300c82e0 0x7fba300c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba30142660 0x7fba30168d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba30169f40 con 0x7fba300ae510 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3016a7e0 con 0x7fba30142660 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300c3510 con 0x7fba300c82e0 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55298/0 (socket says 172.21.15.146:55298) 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 -- 172.21.15.146:0/1170533930 learned_addr learned my addr 172.21.15.146:0/1170533930 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba35800700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba300c82e0 0x7fba300c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36802700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba30142660 0x7fba30168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 -- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba300c82e0 msgr2=0x7fba300c2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba300c82e0 0x7fba300c2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 -- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba30142660 msgr2=0x7fba30168d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba30142660 0x7fba30168d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 -- 172.21.15.146:0/1170533930 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18009420 con 0x7fba300ae510 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba36001700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba2800b610 tx=0x7fba28018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28023070 con 0x7fba300ae510 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba28012b20 con 0x7fba300ae510 2024-08-15T17:00:09.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba28009a20 con 0x7fba300ae510 2024-08-15T17:00:09.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2801e430 con 0x7fba300ae510 2024-08-15T17:00:09.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.821+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3016a470 con 0x7fba300ae510 2024-08-15T17:00:09.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300638f0 con 0x7fba300ae510 2024-08-15T17:00:09.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fba2801e7e0 con 0x7fba300ae510 2024-08-15T17:00:09.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba2dffb700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 0x7fba10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:09.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba2807e4a0 con 0x7fba300ae510 2024-08-15T17:00:09.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba35800700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 0x7fba10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:09.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.825+0000 7fba35800700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 0x7fba10062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba3015db20 tx=0x7fba180091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:09.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.833+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fba2804e850 con 0x7fba300ae510 2024-08-15T17:00:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.997+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba300ca5c0 con 0x7fba300ae510 2024-08-15T17:00:10.010 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:09.997+0000 7fba2dffb700 1 -- 172.21.15.146:0/1170533930 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba28051e70 con 0x7fba300ae510 2024-08-15T17:00:10.010 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:10.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 msgr2=0x7fba10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 0x7fba10062d10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fba3015db20 tx=0x7fba180091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 msgr2=0x7fba300c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba2800b610 tx=0x7fba28018000 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 shutdown_connections 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fba10060860 0x7fba10062d10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fba30142660 0x7fba30168d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fba300c82e0 0x7fba300c2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 --2- 172.21.15.146:0/1170533930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fba300ae510 0x7fba300c2600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 >> 172.21.15.146:0/1170533930 conn(0x7fba300c9830 msgr2=0x7fba30159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 shutdown_connections 2024-08-15T17:00:10.013 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.001+0000 7fba37003700 1 -- 172.21.15.146:0/1170533930 wait complete. 2024-08-15T17:00:10.027 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:10.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.121+0000 7f03ea05e700 1 Processor -- start 2024-08-15T17:00:10.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.121+0000 7f03ea05e700 1 -- start start 2024-08-15T17:00:10.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03e41554c0 0x7f03e40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03e4069ee0 0x7f03e40686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e406b000 con 0x7f03e4155890 2024-08-15T17:00:10.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e406a3f0 con 0x7f03e4069ee0 2024-08-15T17:00:10.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e406a6d0 con 0x7f03e41554c0 2024-08-15T17:00:10.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03e4069ee0 0x7f03e40686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e3fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f03e4155890 0x7f03e4159310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39322/0 (socket says 172.21.15.146:39322) 2024-08-15T17:00:10.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e3fff700 1 -- 172.21.15.146:0/2220328935 learned_addr learned my addr 172.21.15.146:0/2220328935 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:10.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e37fe700 1 --2- 172.21.15.146:0/2220328935 >> v2:172.21.15.161:3300/0 conn(0x7f03e41554c0 0x7f03e40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e2ffd700 1 -- 172.21.15.146:0/2220328935 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645457653 0 0) 0x7f03e406b000 con 0x7f03e4155890 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e2ffd700 1 -- 172.21.15.146:0/2220328935 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d0003570 con 0x7f03e4155890 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 -- 172.21.15.146:0/2220328935 >> v2:172.21.15.161:3300/0 conn(0x7f03e41554c0 msgr2=0x7f03e40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 --2- 172.21.15.146:0/2220328935 >> v2:172.21.15.161:3300/0 conn(0x7f03e41554c0 0x7f03e40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 -- 172.21.15.146:0/2220328935 >> v1:172.21.15.146:6789/0 conn(0x7f03e4155890 legacy=0x7f03e4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 -- 172.21.15.146:0/2220328935 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e406a9b0 con 0x7f03e4069ee0 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03daffd700 1 --2- 172.21.15.146:0/2220328935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e40686a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03e40701a0 tx=0x7f03cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc6a266f68919b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e2ffd700 1 -- 172.21.15.146:0/2220328935 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc011630 con 0x7f03e4069ee0 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e2ffd700 1 -- 172.21.15.146:0/2220328935 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc0117b0 con 0x7f03e4069ee0 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03e2ffd700 1 -- 172.21.15.146:0/2220328935 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc011b70 con 0x7f03e4069ee0 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- 172.21.15.146:0/2220328935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 msgr2=0x7f03e40686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 --2- 172.21.15.146:0/2220328935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e40686a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f03e40701a0 tx=0x7f03cc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- 172.21.15.146:0/2220328935 shutdown_connections 2024-08-15T17:00:10.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 --2- 172.21.15.146:0/2220328935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e40686a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 --2- 172.21.15.146:0/2220328935 >> v2:172.21.15.161:3300/0 conn(0x7f03e41554c0 0x7f03e40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- 172.21.15.146:0/2220328935 >> 172.21.15.146:0/2220328935 conn(0x7f03e40c9bf0 msgr2=0x7f03e406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- 172.21.15.146:0/2220328935 shutdown_connections 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.125+0000 7f03ea05e700 1 -- 172.21.15.146:0/2220328935 wait complete. 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 Processor -- start 2024-08-15T17:00:10.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 -- start start 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e41554c0 0x7f03e41648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e4155890 0x7f03e4152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4158190 con 0x7f03e4069ee0 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4169580 con 0x7f03e41554c0 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03ea05e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4157580 con 0x7f03e4155890 2024-08-15T17:00:10.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55328/0 (socket says 172.21.15.146:55328) 2024-08-15T17:00:10.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 -- 172.21.15.146:0/832176797 learned_addr learned my addr 172.21.15.146:0/832176797 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:10.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e3fff700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e4155890 0x7f03e4152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03daffd700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e41554c0 0x7f03e41648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 -- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e4155890 msgr2=0x7f03e4152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e4155890 0x7f03e4152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 -- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e41554c0 msgr2=0x7f03e41648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e41554c0 0x7f03e41648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 -- 172.21.15.146:0/832176797 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc009420 con 0x7f03e4069ee0 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.129+0000 7f03e37fe700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03d400ea20 tx=0x7f03d400a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400a7a0 con 0x7f03e4069ee0 2024-08-15T17:00:10.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d400ee60 con 0x7f03e4069ee0 2024-08-15T17:00:10.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4157860 con 0x7f03e4069ee0 2024-08-15T17:00:10.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400cb20 con 0x7f03e4069ee0 2024-08-15T17:00:10.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e4157e20 con 0x7f03e4069ee0 2024-08-15T17:00:10.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e40645c0 con 0x7f03e4069ee0 2024-08-15T17:00:10.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f03d40113f0 con 0x7f03e4069ee0 2024-08-15T17:00:10.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03d4015030 con 0x7f03e4069ee0 2024-08-15T17:00:10.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.133+0000 7f03daffd700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.137+0000 7f03daffd700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03e40701a0 tx=0x7f03cc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.155 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.145+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f03d404cd10 con 0x7f03e4069ee0 2024-08-15T17:00:10.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.301+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03e406e870 con 0x7f03e4069ee0 2024-08-15T17:00:10.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.301+0000 7f03dbfff700 1 -- 172.21.15.146:0/832176797 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03d401f020 con 0x7f03e4069ee0 2024-08-15T17:00:10.312 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 msgr2=0x7f03c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f03e40701a0 tx=0x7f03cc017000 comp rx=0 tx=0).stop 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 msgr2=0x7f03e4169040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03d400ea20 tx=0x7f03d400a330 comp rx=0 tx=0).stop 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 shutdown_connections 2024-08-15T17:00:10.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e4155890 0x7f03e4152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e41554c0 0x7f03e41648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 --2- 172.21.15.146:0/832176797 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03e4069ee0 0x7f03e4169040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 >> 172.21.15.146:0/832176797 conn(0x7f03e40c9bf0 msgr2=0x7f03e40c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 shutdown_connections 2024-08-15T17:00:10.317 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.305+0000 7f03ea05e700 1 -- 172.21.15.146:0/832176797 wait complete. 2024-08-15T17:00:10.332 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:10.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 Processor -- start 2024-08-15T17:00:10.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 -- start start 2024-08-15T17:00:10.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb96c15dbc0 0x7fb96c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c0cba90 con 0x7fb96c159300 2024-08-15T17:00:10.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c0b2b60 con 0x7fb96c15dbc0 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb97167c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0b2940 con 0x7fb96c1596d0 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb96affd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb96c159300 0x7fb96c0b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39338/0 (socket says 172.21.15.146:39338) 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb96affd700 1 -- 172.21.15.146:0/538955263 learned_addr learned my addr 172.21.15.146:0/538955263 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.433+0000 7fb96b7fe700 1 --2- 172.21.15.146:0/538955263 >> v2:172.21.15.161:3300/0 conn(0x7fb96c15dbc0 0x7fb96c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb969ffb700 1 -- 172.21.15.146:0/538955263 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336408720 0 0) 0x7fb96c0cba90 con 0x7fb96c159300 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb969ffb700 1 -- 172.21.15.146:0/538955263 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb954003570 con 0x7fb96c159300 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96b7fe700 1 -- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 msgr2=0x7fb96c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96b7fe700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96b7fe700 1 -- 172.21.15.146:0/538955263 >> v1:172.21.15.146:6789/0 conn(0x7fb96c159300 legacy=0x7fb96c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96b7fe700 1 -- 172.21.15.146:0/538955263 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0cb160 con 0x7fb96c15dbc0 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:10.448 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb96b7fe700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c067340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb964009e60 tx=0x7fb964009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68ff924921bfa6f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb969ffb700 1 -- 172.21.15.146:0/538955263 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400e070 con 0x7fb96c15dbc0 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb969ffb700 1 -- 172.21.15.146:0/538955263 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96400bab0 con 0x7fb96c15dbc0 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb969ffb700 1 -- 172.21.15.146:0/538955263 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96400be30 con 0x7fb96c15dbc0 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 msgr2=0x7fb96c067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c067340 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb964009e60 tx=0x7fb964009780 comp rx=0 tx=0).stop 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- 172.21.15.146:0/538955263 shutdown_connections 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 --2- 172.21.15.146:0/538955263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c067340 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- 172.21.15.146:0/538955263 >> 172.21.15.146:0/538955263 conn(0x7fb96c0682c0 msgr2=0x7fb96c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- 172.21.15.146:0/538955263 shutdown_connections 2024-08-15T17:00:10.450 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- 172.21.15.146:0/538955263 wait complete. 2024-08-15T17:00:10.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 Processor -- start 2024-08-15T17:00:10.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.437+0000 7fb97167c700 1 -- start start 2024-08-15T17:00:10.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c15c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c16d3d0 con 0x7fb96c159300 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c16c7c0 con 0x7fb96c15dbc0 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c16caa0 con 0x7fb96c1596d0 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55338/0 (socket says 172.21.15.146:55338) 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 -- 172.21.15.146:0/1367116494 learned_addr learned my addr 172.21.15.146:0/1367116494 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96b7fe700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c15c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 -- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 msgr2=0x7fb96c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 -- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 msgr2=0x7fb96c15c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c15c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 -- 172.21.15.146:0/1367116494 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c009420 con 0x7fb96c159300 2024-08-15T17:00:10.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb96affd700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9600011e0 tx=0x7fb960007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960005b30 con 0x7fb96c159300 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb960005c90 con 0x7fb96c159300 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964009a20 con 0x7fb96c159300 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960007320 con 0x7fb96c159300 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c16d060 con 0x7fb96c159300 2024-08-15T17:00:10.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.441+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c0645c0 con 0x7fb96c159300 2024-08-15T17:00:10.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.445+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb960007690 con 0x7fb96c159300 2024-08-15T17:00:10.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.445+0000 7fb952ffd700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 0x7fb958062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.445+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb96007c070 con 0x7fb96c159300 2024-08-15T17:00:10.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.445+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 0x7fb958062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.445+0000 7fb96a7fc700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 0x7fb958062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0093f0 tx=0x7fb95c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.453+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb96004c4c0 con 0x7fb96c159300 2024-08-15T17:00:10.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.617+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb96c0caac0 con 0x7fb96c159300 2024-08-15T17:00:10.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.617+0000 7fb952ffd700 1 -- 172.21.15.146:0/1367116494 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb960017020 con 0x7fb96c159300 2024-08-15T17:00:10.629 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 msgr2=0x7fb958062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 0x7fb958062d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0093f0 tx=0x7fb95c008040 comp rx=0 tx=0).stop 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 msgr2=0x7fb96c16be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb9600011e0 tx=0x7fb960007df0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 shutdown_connections 2024-08-15T17:00:10.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb958060860 0x7fb958062d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb96c15dbc0 0x7fb96c15c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb96c1596d0 0x7fb96c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 --2- 172.21.15.146:0/1367116494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb96c159300 0x7fb96c16be30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 >> 172.21.15.146:0/1367116494 conn(0x7fb96c0682c0 msgr2=0x7fb96c0ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 shutdown_connections 2024-08-15T17:00:10.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.621+0000 7fb97167c700 1 -- 172.21.15.146:0/1367116494 wait complete. 2024-08-15T17:00:10.644 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:10.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.733+0000 7f284ec6d700 1 Processor -- start 2024-08-15T17:00:10.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.733+0000 7f284ec6d700 1 -- start start 2024-08-15T17:00:10.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f2848164060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480aa270 con 0x7f2848068c10 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480aa550 con 0x7f2848068840 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2848069090 con 0x7f2848159170 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284d46a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2848068c10 0x7f2848158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39348/0 (socket says 172.21.15.146:39348) 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284d46a700 1 -- 172.21.15.146:0/4256765152 learned_addr learned my addr 172.21.15.146:0/4256765152 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:10.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284e46c700 1 --2- 172.21.15.146:0/4256765152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f2848164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284cc69700 1 -- 172.21.15.146:0/4256765152 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143633621 0 0) 0x7f28480aa270 con 0x7f2848068c10 2024-08-15T17:00:10.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284cc69700 1 -- 172.21.15.146:0/4256765152 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2834003570 con 0x7f2848068c10 2024-08-15T17:00:10.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284e46c700 1 -- 172.21.15.146:0/4256765152 >> v1:172.21.15.161:6789/0 conn(0x7f2848068840 legacy=0x7f28480a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284e46c700 1 -- 172.21.15.146:0/4256765152 >> v1:172.21.15.146:6789/0 conn(0x7f2848068c10 legacy=0x7f2848158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284e46c700 1 -- 172.21.15.146:0/4256765152 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480aa830 con 0x7f2848159170 2024-08-15T17:00:10.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284e46c700 1 --2- 172.21.15.146:0/4256765152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f2848164060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2840009ec0 tx=0x7f2840009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fca9d943f349aa1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284cc69700 1 -- 172.21.15.146:0/4256765152 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284000e070 con 0x7f2848159170 2024-08-15T17:00:10.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284cc69700 1 -- 172.21.15.146:0/4256765152 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284000bbf0 con 0x7f2848159170 2024-08-15T17:00:10.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284cc69700 1 -- 172.21.15.146:0/4256765152 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2840012430 con 0x7f2848159170 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- 172.21.15.146:0/4256765152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 msgr2=0x7f2848164060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 --2- 172.21.15.146:0/4256765152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f2848164060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2840009ec0 tx=0x7f2840009780 comp rx=0 tx=0).stop 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- 172.21.15.146:0/4256765152 shutdown_connections 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 --2- 172.21.15.146:0/4256765152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f2848164060 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.737+0000 7f284ec6d700 1 -- 172.21.15.146:0/4256765152 >> 172.21.15.146:0/4256765152 conn(0x7f284806d800 msgr2=0x7f284806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- 172.21.15.146:0/4256765152 shutdown_connections 2024-08-15T17:00:10.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- 172.21.15.146:0/4256765152 wait complete. 2024-08-15T17:00:10.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 Processor -- start 2024-08-15T17:00:10.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- start start 2024-08-15T17:00:10.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848068840 0x7f28480caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f284815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480cb420 con 0x7f2848068c10 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848168ea0 con 0x7f2848068840 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284ec6d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480c1d50 con 0x7f2848159170 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284d46a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284d46a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55340/0 (socket says 172.21.15.146:55340) 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284d46a700 1 -- 172.21.15.146:0/1037219549 learned_addr learned my addr 172.21.15.146:0/1037219549 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284dc6b700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848068840 0x7f28480caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.741+0000 7f284e46c700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f284815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 -- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 msgr2=0x7f284815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f284815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 -- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848068840 msgr2=0x7f28480caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848068840 0x7f28480caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 -- 172.21.15.146:0/1037219549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840009a20 con 0x7f2848068c10 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284e46c700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f284815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284d46a700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f283800c7c0 tx=0x7f2838007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838005b30 con 0x7f2848068c10 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2838005cb0 con 0x7f2848068c10 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28480c2030 con 0x7f2848068c10 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2838007320 con 0x7f2848068c10 2024-08-15T17:00:10.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28480c25f0 con 0x7f2848068c10 2024-08-15T17:00:10.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28480638f0 con 0x7f2848068c10 2024-08-15T17:00:10.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f283801b3b0 con 0x7f2848068c10 2024-08-15T17:00:10.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 0x7f282c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:10.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f283807c090 con 0x7f2848068c10 2024-08-15T17:00:10.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.745+0000 7f284dc6b700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 0x7f282c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:10.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.749+0000 7f284dc6b700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 0x7f282c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28440039e0 tx=0x7f2844007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.753+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f283804c4e0 con 0x7f2848068c10 2024-08-15T17:00:10.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.917+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f284806e590 con 0x7f2848068c10 2024-08-15T17:00:10.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f283dffb700 1 -- 172.21.15.146:0/1037219549 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2838055030 con 0x7f2848068c10 2024-08-15T17:00:10.931 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:10.931 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 msgr2=0x7f282c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 0x7f282c062d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f28440039e0 tx=0x7f2844007ec0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 msgr2=0x7f28480c1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f283800c7c0 tx=0x7f2838007e00 comp rx=0 tx=0).stop 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 shutdown_connections 2024-08-15T17:00:10.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f282c060860 0x7f282c062d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2848159170 0x7f284815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2848068c10 0x7f28480c1370 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 --2- 172.21.15.146:0/1037219549 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2848068840 0x7f28480caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 >> 172.21.15.146:0/1037219549 conn(0x7f284806d800 msgr2=0x7f28480aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 shutdown_connections 2024-08-15T17:00:10.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:10.921+0000 7f284ec6d700 1 -- 172.21.15.146:0/1037219549 wait complete. 2024-08-15T17:00:10.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:10.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:11.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.045+0000 7fe7e0ec9700 1 Processor -- start 2024-08-15T17:00:11.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.045+0000 7fe7e0ec9700 1 -- start start 2024-08-15T17:00:11.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc0aab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7dc068ca0 0x7fe7dc066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc067420 con 0x7fe7dc158b70 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc0ab0f0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0ab3d0 con 0x7fe7dc0688d0 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7dc068ca0 0x7fe7dc066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe7dc068ca0 0x7fe7dc066ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55350/0 (socket says 172.21.15.146:55350) 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/3837710455 learned_addr learned my addr 172.21.15.146:0/3837710455 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7db7fe700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc0aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/3837710455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 msgr2=0x7fe7dc0aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc0aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/3837710455 >> v1:172.21.15.161:6789/0 conn(0x7fe7dc158b70 legacy=0x7fe7dc158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/3837710455 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc0ab6b0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7d2ffd700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc066ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40099b0 tx=0x7fe7c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142762477aad557e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7daffd700 1 -- 172.21.15.146:0/3837710455 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c400b860 con 0x7fe7dc068ca0 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7daffd700 1 -- 172.21.15.146:0/3837710455 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c400b9c0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7daffd700 1 -- 172.21.15.146:0/3837710455 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c400bd60 con 0x7fe7dc068ca0 2024-08-15T17:00:11.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/3837710455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 msgr2=0x7fe7dc066ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc066ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40099b0 tx=0x7fe7c4009250 comp rx=0 tx=0).stop 2024-08-15T17:00:11.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/3837710455 shutdown_connections 2024-08-15T17:00:11.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc066ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/3837710455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc0aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/3837710455 >> 172.21.15.146:0/3837710455 conn(0x7fe7dc06e310 msgr2=0x7fe7dc06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/3837710455 shutdown_connections 2024-08-15T17:00:11.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.049+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/3837710455 wait complete. 2024-08-15T17:00:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 Processor -- start 2024-08-15T17:00:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- start start 2024-08-15T17:00:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7dc158b70 0x7fe7dc0a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc1691e0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc158720 con 0x7fe7dc158b70 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0c1ea0 con 0x7fe7dc0688d0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55366/0 (socket says 172.21.15.146:55366) 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/1186023684 learned_addr learned my addr 172.21.15.146:0/1186023684 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7db7fe700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7dbfff700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7dc158b70 0x7fe7dc0a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 msgr2=0x7fe7dc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7dc158b70 msgr2=0x7fe7dc0a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7dc158b70 0x7fe7dc0a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 -- 172.21.15.146:0/1186023684 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40094f0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d2ffd700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc064ce0 tx=0x7fe7c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4011e40 con 0x7fe7dc068ca0 2024-08-15T17:00:11.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4024440 con 0x7fe7dc068ca0 2024-08-15T17:00:11.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc0c2180 con 0x7fe7dc068ca0 2024-08-15T17:00:11.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40247e0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.053+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc0c2740 con 0x7fe7dc068ca0 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc169350 con 0x7fe7dc068ca0 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe7c401a420 con 0x7fe7dc068ca0 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7d3fff700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 0x7fe7c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe7c407ea00 con 0x7fe7dc068ca0 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7db7fe700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 0x7fe7c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.057+0000 7fe7db7fe700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 0x7fe7c0062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006fd0 tx=0x7fe7cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.065+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe7c404eed0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.237 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.225+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7dc0638f0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.225+0000 7fe7d3fff700 1 -- 172.21.15.146:0/1186023684 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7c40524f0 con 0x7fe7dc068ca0 2024-08-15T17:00:11.238 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:11.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 msgr2=0x7fe7c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 0x7fe7c0062d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006fd0 tx=0x7fe7cc008040 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 msgr2=0x7fe7dc0c1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc064ce0 tx=0x7fe7c4017000 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 shutdown_connections 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe7c0060860 0x7fe7c0062d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7dc158b70 0x7fe7dc0a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe7dc068ca0 0x7fe7dc0c1530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 --2- 172.21.15.146:0/1186023684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe7dc0688d0 0x7fe7dc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 >> 172.21.15.146:0/1186023684 conn(0x7fe7dc06e310 msgr2=0x7fe7dc06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 shutdown_connections 2024-08-15T17:00:11.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.229+0000 7fe7e0ec9700 1 -- 172.21.15.146:0/1186023684 wait complete. 2024-08-15T17:00:11.252 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:11.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.353+0000 7f9f23fff700 1 Processor -- start 2024-08-15T17:00:11.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.353+0000 7f9f23fff700 1 -- start start 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 0x7f9f2406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 0x7f9f24155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f240ace30 con 0x7f9f24152bd0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f240b1c80 con 0x7f9f240b1830 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24156dd0 con 0x7f9f2406ead0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f237fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9f24152bd0 0x7f9f24153010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39352/0 (socket says 172.21.15.146:39352) 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f237fe700 1 -- 172.21.15.146:0/2039787911 learned_addr learned my addr 172.21.15.146:0/2039787911 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/2039787911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 0x7f9f2406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f227fc700 1 --2- 172.21.15.146:0/2039787911 >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 0x7f9f24155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f21ffb700 1 -- 172.21.15.146:0/2039787911 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501075908 0 0) 0x7f9f240ace30 con 0x7f9f24152bd0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f21ffb700 1 -- 172.21.15.146:0/2039787911 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f04003570 con 0x7f9f24152bd0 2024-08-15T17:00:11.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 -- 172.21.15.146:0/2039787911 >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 msgr2=0x7f9f24155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/2039787911 >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 0x7f9f24155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 -- 172.21.15.146:0/2039787911 >> v1:172.21.15.146:6789/0 conn(0x7f9f24152bd0 legacy=0x7f9f24153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 -- 172.21.15.146:0/2039787911 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f24156450 con 0x7f9f2406ead0 2024-08-15T17:00:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f227fc700 1 --2- 172.21.15.146:0/2039787911 >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 0x7f9f24155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:11.370 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/2039787911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 0x7f9f2406eea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f140098e0 tx=0x7f9f140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03a889264d133ce server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f21ffb700 1 -- 172.21.15.146:0/2039787911 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011630 con 0x7f9f2406ead0 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f21ffb700 1 -- 172.21.15.146:0/2039787911 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14011790 con 0x7f9f2406ead0 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f21ffb700 1 -- 172.21.15.146:0/2039787911 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011b10 con 0x7f9f2406ead0 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- 172.21.15.146:0/2039787911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 msgr2=0x7f9f2406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 --2- 172.21.15.146:0/2039787911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 0x7f9f2406eea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f140098e0 tx=0x7f9f140091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- 172.21.15.146:0/2039787911 shutdown_connections 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 --2- 172.21.15.146:0/2039787911 >> v2:172.21.15.161:3300/0 conn(0x7f9f240b1830 0x7f9f24155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.371 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 --2- 172.21.15.146:0/2039787911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f2406ead0 0x7f9f2406eea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- 172.21.15.146:0/2039787911 >> 172.21.15.146:0/2039787911 conn(0x7f9f240678a0 msgr2=0x7f9f24067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- 172.21.15.146:0/2039787911 shutdown_connections 2024-08-15T17:00:11.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.357+0000 7f9f23fff700 1 -- 172.21.15.146:0/2039787911 wait complete. 2024-08-15T17:00:11.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.361+0000 7f9f23fff700 1 Processor -- start 2024-08-15T17:00:11.373 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.361+0000 7f9f23fff700 1 -- start start 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f2406ead0 0x7f9f240a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f240b1830 0x7f9f240af300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2416b890 con 0x7f9f24152bd0 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24143410 con 0x7f9f2406ead0 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f241525e0 con 0x7f9f240b1830 2024-08-15T17:00:11.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55370/0 (socket says 172.21.15.146:55370) 2024-08-15T17:00:11.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 -- 172.21.15.146:0/931734741 learned_addr learned my addr 172.21.15.146:0/931734741 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f2406ead0 0x7f9f240a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f227fc700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f240b1830 0x7f9f240af300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 -- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f240b1830 msgr2=0x7f9f240af300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f240b1830 0x7f9f240af300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 -- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f2406ead0 msgr2=0x7f9f240a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f2406ead0 0x7f9f240a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 -- 172.21.15.146:0/931734741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c009420 con 0x7f9f24152bd0 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f237fe700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00cc50 tx=0x7f9f1c001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00a870 con 0x7f9f24152bd0 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c00a9d0 con 0x7f9f24152bd0 2024-08-15T17:00:11.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f14009420 con 0x7f9f24152bd0 2024-08-15T17:00:11.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00ad90 con 0x7f9f24152bd0 2024-08-15T17:00:11.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.365+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24152300 con 0x7f9f24152bd0 2024-08-15T17:00:11.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f24064330 con 0x7f9f24152bd0 2024-08-15T17:00:11.379 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9f1c007300 con 0x7f9f24152bd0 2024-08-15T17:00:11.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f1affd700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 0x7f9f08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f1c07cec0 con 0x7f9f24152bd0 2024-08-15T17:00:11.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 0x7f9f08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.369+0000 7f9f22ffd700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 0x7f9f08062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f140091e0 tx=0x7f9f14017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.377+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9f1c04d310 con 0x7f9f24152bd0 2024-08-15T17:00:11.550 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.537+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f24143550 con 0x7f9f24152bd0 2024-08-15T17:00:11.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f1affd700 1 -- 172.21.15.146:0/931734741 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f1c050930 con 0x7f9f24152bd0 2024-08-15T17:00:11.551 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 msgr2=0x7f9f08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 0x7f9f08062d10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f140091e0 tx=0x7f9f14017000 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 msgr2=0x7f9f241438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00cc50 tx=0x7f9f1c001a60 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 shutdown_connections 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f08060860 0x7f9f08062d10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f24152bd0 0x7f9f241438a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f240b1830 0x7f9f240af300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 --2- 172.21.15.146:0/931734741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f2406ead0 0x7f9f240a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 >> 172.21.15.146:0/931734741 conn(0x7f9f240678a0 msgr2=0x7f9f24068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 shutdown_connections 2024-08-15T17:00:11.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.541+0000 7f9f23fff700 1 -- 172.21.15.146:0/931734741 wait complete. 2024-08-15T17:00:11.565 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:11.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.669+0000 7f91df415700 1 Processor -- start 2024-08-15T17:00:11.683 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.669+0000 7f91df415700 1 -- start start 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91df415700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 0x7f91d00081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91df415700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f91d00396f0 0x7f91d0055a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91df415700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d00dfee0 con 0x7f91d0007e10 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91df415700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00e0130 con 0x7f91d00396f0 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91df415700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d004ac40 con 0x7f91d0056920 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91de413700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f91d0007e10 0x7f91d0056210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39356/0 (socket says 172.21.15.146:39356) 2024-08-15T17:00:11.685 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91de413700 1 -- 172.21.15.146:0/140076351 learned_addr learned my addr 172.21.15.146:0/140076351 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 --2- 172.21.15.146:0/140076351 >> v2:172.21.15.161:3300/0 conn(0x7f91d00396f0 0x7f91d0055a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91ddc12700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 0x7f91d00081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dd411700 1 -- 172.21.15.146:0/140076351 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24134699 0 0) 0x7f91d00dfee0 con 0x7f91d0007e10 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dd411700 1 -- 172.21.15.146:0/140076351 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c4003570 con 0x7f91d0007e10 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 -- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 msgr2=0x7f91d00081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 0x7f91d00081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 -- 172.21.15.146:0/140076351 >> v1:172.21.15.146:6789/0 conn(0x7f91d0007e10 legacy=0x7f91d0056210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 -- 172.21.15.146:0/140076351 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d004af20 con 0x7f91d00396f0 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91ddc12700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 0x7f91d00081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:11.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dec14700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d00396f0 0x7f91d0055a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91d8065b20 tx=0x7f91d8064ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c76ab23b3e2ac52 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dd411700 1 -- 172.21.15.146:0/140076351 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d804c190 con 0x7f91d00396f0 2024-08-15T17:00:11.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dd411700 1 -- 172.21.15.146:0/140076351 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8064870 con 0x7f91d00396f0 2024-08-15T17:00:11.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.673+0000 7f91dd411700 1 -- 172.21.15.146:0/140076351 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8064c30 con 0x7f91d00396f0 2024-08-15T17:00:11.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d00396f0 msgr2=0x7f91d0055a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d00396f0 0x7f91d0055a60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f91d8065b20 tx=0x7f91d8064ee0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- 172.21.15.146:0/140076351 shutdown_connections 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d0056920 0x7f91d00081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 --2- 172.21.15.146:0/140076351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d00396f0 0x7f91d0055a60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- 172.21.15.146:0/140076351 >> 172.21.15.146:0/140076351 conn(0x7f91d0038be0 msgr2=0x7f91d0038fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- 172.21.15.146:0/140076351 shutdown_connections 2024-08-15T17:00:11.689 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- 172.21.15.146:0/140076351 wait complete. 2024-08-15T17:00:11.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 Processor -- start 2024-08-15T17:00:11.690 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 -- start start 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.677+0000 7f91df415700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 0x7f91d00c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d00396f0 0x7f91d00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0033220 con 0x7f91d0056920 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0034030 con 0x7f91d0007e10 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d000c870 con 0x7f91d00396f0 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55380/0 (socket says 172.21.15.146:55380) 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 -- 172.21.15.146:0/1183763755 learned_addr learned my addr 172.21.15.146:0/1183763755 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91de413700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 0x7f91d00c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ddc12700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d00396f0 0x7f91d00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 -- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d00396f0 msgr2=0x7f91d00ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d00396f0 0x7f91d00ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 -- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 msgr2=0x7f91d00c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 0x7f91d00c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 -- 172.21.15.146:0/1183763755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c8009420 con 0x7f91d0056920 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91de413700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 0x7f91d00c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:11.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91dec14700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f91d8065940 tx=0x7f91d806f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8096070 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d0033420 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d806d070 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d804c190 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d00339e0 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d00341c0 con 0x7f91d0056920 2024-08-15T17:00:11.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f91d80696e0 con 0x7f91d0056920 2024-08-15T17:00:11.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.681+0000 7f91ce7fc700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 0x7f91bc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.685+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91d80f3e80 con 0x7f91d0056920 2024-08-15T17:00:11.695 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.685+0000 7f91de413700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 0x7f91bc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.685+0000 7f91de413700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 0x7f91bc062d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91d4001360 tx=0x7f91d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.685+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f91d80c42d0 con 0x7f91d0056920 2024-08-15T17:00:11.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.853+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91d0005800 con 0x7f91d0056920 2024-08-15T17:00:11.864 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.853+0000 7f91ce7fc700 1 -- 172.21.15.146:0/1183763755 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91d80c78f0 con 0x7f91d0056920 2024-08-15T17:00:11.864 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:11.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 msgr2=0x7f91bc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 0x7f91bc062d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91d4001360 tx=0x7f91d4009040 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 msgr2=0x7f91d000b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f91d8065940 tx=0x7f91d806f000 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 shutdown_connections 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91bc060890 0x7f91bc062d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91d0056920 0x7f91d000b1b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91d00396f0 0x7f91d00ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 --2- 172.21.15.146:0/1183763755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91d0007e10 0x7f91d00c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 >> 172.21.15.146:0/1183763755 conn(0x7f91d0038be0 msgr2=0x7f91d00081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 shutdown_connections 2024-08-15T17:00:11.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.857+0000 7f91df415700 1 -- 172.21.15.146:0/1183763755 wait complete. 2024-08-15T17:00:11.884 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:11.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 Processor -- start 2024-08-15T17:00:11.988 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 -- start start 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 0x7ffbdc0cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffbdc0d0010 0x7ffbdc0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbdc0cf190 con 0x7ffbdc15cea0 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc15d9f0 con 0x7ffbdc0d0010 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdbfff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbdc0d09a0 con 0x7ffbdc159040 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffbdc0d0010 0x7ffbdc0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ffbdc0d0010 0x7ffbdc0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55394/0 (socket says 172.21.15.146:55394) 2024-08-15T17:00:11.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdb7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ffbdc15cea0 0x7ffbdc15d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39368/0 (socket says 172.21.15.146:39368) 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 -- 172.21.15.146:0/3288762821 learned_addr learned my addr 172.21.15.146:0/3288762821 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdaffd700 1 --2- 172.21.15.146:0/3288762821 >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 0x7ffbdc0cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 -- 172.21.15.146:0/3288762821 >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 msgr2=0x7ffbdc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 --2- 172.21.15.146:0/3288762821 >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 0x7ffbdc0cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbd9ffb700 1 -- 172.21.15.146:0/3288762821 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841882025 0 0) 0x7ffbdc0cf190 con 0x7ffbdc15cea0 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 -- 172.21.15.146:0/3288762821 >> v1:172.21.15.146:6789/0 conn(0x7ffbdc15cea0 legacy=0x7ffbdc15d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 -- 172.21.15.146:0/3288762821 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbdc0ce5e0 con 0x7ffbdc0d0010 2024-08-15T17:00:11.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbdaffd700 1 --2- 172.21.15.146:0/3288762821 >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 0x7ffbdc0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:11.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbda7fc700 1 --2- 172.21.15.146:0/3288762821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0d0460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc070d70 tx=0x7ffbc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b25534bf29d5d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbd9ffb700 1 -- 172.21.15.146:0/3288762821 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400b860 con 0x7ffbdc0d0010 2024-08-15T17:00:11.992 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbd9ffb700 1 -- 172.21.15.146:0/3288762821 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400b9c0 con 0x7ffbdc0d0010 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.977+0000 7ffbd9ffb700 1 -- 172.21.15.146:0/3288762821 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400bd60 con 0x7ffbdc0d0010 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- 172.21.15.146:0/3288762821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 msgr2=0x7ffbdc0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/3288762821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0d0460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc070d70 tx=0x7ffbc4009250 comp rx=0 tx=0).stop 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- 172.21.15.146:0/3288762821 shutdown_connections 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/3288762821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0d0460 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/3288762821 >> v2:172.21.15.161:3300/0 conn(0x7ffbdc159040 0x7ffbdc0cf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- 172.21.15.146:0/3288762821 >> 172.21.15.146:0/3288762821 conn(0x7ffbdc068250 msgr2=0x7ffbdc068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- 172.21.15.146:0/3288762821 shutdown_connections 2024-08-15T17:00:11.993 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- 172.21.15.146:0/3288762821 wait complete. 2024-08-15T17:00:11.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 Processor -- start 2024-08-15T17:00:11.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.981+0000 7ffbdbfff700 1 -- start start 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbdc159040 0x7ffbdc16aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 0x7ffbdc0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc169980 con 0x7ffbdc0d0010 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0afe70 con 0x7ffbdc159040 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc0b0150 con 0x7ffbdc15cea0 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55396/0 (socket says 172.21.15.146:55396) 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 -- 172.21.15.146:0/1110149056 learned_addr learned my addr 172.21.15.146:0/1110149056 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdb7fe700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 0x7ffbdc0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbda7fc700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbdc159040 0x7ffbdc16aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 -- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 msgr2=0x7ffbdc0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 0x7ffbdc0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 -- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbdc159040 msgr2=0x7ffbdc16aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbdc159040 0x7ffbdc16aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 -- 172.21.15.146:0/1110149056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40094f0 con 0x7ffbdc0d0010 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdb7fe700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 0x7ffbdc0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdaffd700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00d860 tx=0x7ffbcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0125f0 con 0x7ffbdc0d0010 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc012750 con 0x7ffbdc0d0010 2024-08-15T17:00:11.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc0b0430 con 0x7ffbdc0d0010 2024-08-15T17:00:12.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc012af0 con 0x7ffbdc0d0010 2024-08-15T17:00:12.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.985+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc0ae760 con 0x7ffbdc0d0010 2024-08-15T17:00:12.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.989+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc0643a0 con 0x7ffbdc0d0010 2024-08-15T17:00:12.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.989+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ffbcc013480 con 0x7ffbdc0d0010 2024-08-15T17:00:12.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.989+0000 7ffbd2ffd700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 0x7ffbbc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.989+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffbcc07cd60 con 0x7ffbdc0d0010 2024-08-15T17:00:12.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.989+0000 7ffbda7fc700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 0x7ffbbc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.993+0000 7ffbda7fc700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 0x7ffbbc062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc070d70 tx=0x7ffbc4017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:11.997+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ffbcc04d1b0 con 0x7ffbdc0d0010 2024-08-15T17:00:12.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.161+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbdc0cd320 con 0x7ffbdc0d0010 2024-08-15T17:00:12.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.161+0000 7ffbd2ffd700 1 -- 172.21.15.146:0/1110149056 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7ffbcc0507d0 con 0x7ffbdc0d0010 2024-08-15T17:00:12.173 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:12.173 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:12.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 msgr2=0x7ffbbc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 0x7ffbbc062d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc070d70 tx=0x7ffbc4017000 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 msgr2=0x7ffbdc0ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00d860 tx=0x7ffbcc008040 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 shutdown_connections 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ffbbc060860 0x7ffbbc062d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffbdc15cea0 0x7ffbdc0b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffbdc159040 0x7ffbdc16aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 --2- 172.21.15.146:0/1110149056 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbdc0d0010 0x7ffbdc0ae220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 >> 172.21.15.146:0/1110149056 conn(0x7ffbdc068250 msgr2=0x7ffbdc152bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 shutdown_connections 2024-08-15T17:00:12.176 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.165+0000 7ffbdbfff700 1 -- 172.21.15.146:0/1110149056 wait complete. 2024-08-15T17:00:12.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:12.187 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:12.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.285+0000 7f8a6688a700 1 Processor -- start 2024-08-15T17:00:12.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.285+0000 7f8a6688a700 1 -- start start 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a6013fa50 0x7f8a600aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6006e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600b0810 con 0x7f8a6013f680 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a600afc70 con 0x7f8a6013fa50 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600af060 con 0x7f8a600ad6c0 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65888700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8a6013f680 0x7f8a600b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39384/0 (socket says 172.21.15.146:39384) 2024-08-15T17:00:12.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65888700 1 -- 172.21.15.146:0/3088976778 learned_addr learned my addr 172.21.15.146:0/3088976778 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 --2- 172.21.15.146:0/3088976778 >> v2:172.21.15.161:3300/0 conn(0x7f8a6013fa50 0x7f8a600aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a66089700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6006e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a64886700 1 -- 172.21.15.146:0/3088976778 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443127729 0 0) 0x7f8a600b0810 con 0x7f8a6013f680 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a64886700 1 -- 172.21.15.146:0/3088976778 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a48003570 con 0x7f8a6013f680 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 -- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 msgr2=0x7f8a6006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6006e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 -- 172.21.15.146:0/3088976778 >> v1:172.21.15.146:6789/0 conn(0x7f8a6013f680 legacy=0x7f8a600b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 -- 172.21.15.146:0/3088976778 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600af340 con 0x7f8a6013fa50 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a65087700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013fa50 0x7f8a600aea80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098e0 tx=0x7f8a540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cb9286d2d3627ae server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a64886700 1 -- 172.21.15.146:0/3088976778 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54011630 con 0x7f8a6013fa50 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a64886700 1 -- 172.21.15.146:0/3088976778 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a54011790 con 0x7f8a6013fa50 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a64886700 1 -- 172.21.15.146:0/3088976778 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54011b30 con 0x7f8a6013fa50 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013fa50 msgr2=0x7f8a600aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013fa50 0x7f8a600aea80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098e0 tx=0x7f8a540091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- 172.21.15.146:0/3088976778 shutdown_connections 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6006e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 --2- 172.21.15.146:0/3088976778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013fa50 0x7f8a600aea80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- 172.21.15.146:0/3088976778 >> 172.21.15.146:0/3088976778 conn(0x7f8a600678a0 msgr2=0x7f8a60067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- 172.21.15.146:0/3088976778 shutdown_connections 2024-08-15T17:00:12.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.289+0000 7f8a6688a700 1 -- 172.21.15.146:0/3088976778 wait complete. 2024-08-15T17:00:12.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 Processor -- start 2024-08-15T17:00:12.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 -- start start 2024-08-15T17:00:12.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6012e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013f680 0x7f8a601534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a601580b0 con 0x7f8a60069540 2024-08-15T17:00:12.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6006a5e0 con 0x7f8a6013f680 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a6688a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60157a60 con 0x7f8a600ad6c0 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55406/0 (socket says 172.21.15.146:55406) 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 -- 172.21.15.146:0/2228255675 learned_addr learned my addr 172.21.15.146:0/2228255675 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a65888700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6012e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a65087700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013f680 0x7f8a601534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 -- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 msgr2=0x7f8a6012e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6012e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 -- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013f680 msgr2=0x7f8a601534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013f680 0x7f8a601534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 -- 172.21.15.146:0/2228255675 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009420 con 0x7f8a60069540 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a65888700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6012e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:12.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.293+0000 7f8a66089700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800ea50 tx=0x7f8a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800c850 con 0x7f8a60069540 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5800c9d0 con 0x7f8a60069540 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a60157d40 con 0x7f8a60069540 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800cd90 con 0x7f8a60069540 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a601502f0 con 0x7f8a60069540 2024-08-15T17:00:12.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a601578c0 con 0x7f8a60069540 2024-08-15T17:00:12.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8a58013680 con 0x7f8a60069540 2024-08-15T17:00:12.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 0x7f8a4c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a5807cdf0 con 0x7f8a60069540 2024-08-15T17:00:12.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.297+0000 7f8a65888700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 0x7f8a4c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.301+0000 7f8a65888700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 0x7f8a4c062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8a50001580 tx=0x7f8a50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.305+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8a601578c0 con 0x7f8a60069540 2024-08-15T17:00:12.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.469+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a601578c0 con 0x7f8a60069540 2024-08-15T17:00:12.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.469+0000 7f8a5dffb700 1 -- 172.21.15.146:0/2228255675 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a58017020 con 0x7f8a60069540 2024-08-15T17:00:12.481 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:12.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 msgr2=0x7f8a4c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 0x7f8a4c062d10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8a50001580 tx=0x7f8a50009040 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 msgr2=0x7f8a60158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800ea50 tx=0x7f8a580091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 shutdown_connections 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8a4c060860 0x7f8a4c062d10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a60069540 0x7f8a60158540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a6013f680 0x7f8a601534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 --2- 172.21.15.146:0/2228255675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a600ad6c0 0x7f8a6012e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 >> 172.21.15.146:0/2228255675 conn(0x7f8a600678a0 msgr2=0x7f8a60063550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 shutdown_connections 2024-08-15T17:00:12.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.473+0000 7f8a6688a700 1 -- 172.21.15.146:0/2228255675 wait complete. 2024-08-15T17:00:12.496 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:12.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 Processor -- start 2024-08-15T17:00:12.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 -- start start 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad9c1590b0 0x7fad9c066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad9c159480 0x7fad9c067340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c0ce210 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0cba00 con 0x7fad9c159480 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9b59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c0cdc90 con 0x7fad9c1590b0 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad99d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad9c159480 0x7fad9c067340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9ad9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fad9c0cd2b0 0x7fad9c15ce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39390/0 (socket says 172.21.15.146:39390) 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9ad9d700 1 -- 172.21.15.146:0/1862502892 learned_addr learned my addr 172.21.15.146:0/1862502892 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.593+0000 7fad9a59c700 1 --2- 172.21.15.146:0/1862502892 >> v2:172.21.15.161:3300/0 conn(0x7fad9c1590b0 0x7fad9c066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9959a700 1 -- 172.21.15.146:0/1862502892 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714402728 0 0) 0x7fad9c0ce210 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9959a700 1 -- 172.21.15.146:0/1862502892 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad84003570 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad99d9b700 1 -- 172.21.15.146:0/1862502892 >> v2:172.21.15.161:3300/0 conn(0x7fad9c1590b0 msgr2=0x7fad9c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad99d9b700 1 --2- 172.21.15.146:0/1862502892 >> v2:172.21.15.161:3300/0 conn(0x7fad9c1590b0 0x7fad9c066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad99d9b700 1 -- 172.21.15.146:0/1862502892 >> v1:172.21.15.146:6789/0 conn(0x7fad9c0cd2b0 legacy=0x7fad9c15ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad99d9b700 1 -- 172.21.15.146:0/1862502892 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c0cb440 con 0x7fad9c159480 2024-08-15T17:00:12.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad99d9b700 1 --2- 172.21.15.146:0/1862502892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c159480 0x7fad9c067340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad9c071780 tx=0x7fad8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcb0e73711ab9d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9959a700 1 -- 172.21.15.146:0/1862502892 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c011630 con 0x7fad9c159480 2024-08-15T17:00:12.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9959a700 1 -- 172.21.15.146:0/1862502892 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c0117b0 con 0x7fad9c159480 2024-08-15T17:00:12.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9959a700 1 -- 172.21.15.146:0/1862502892 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c011b70 con 0x7fad9c159480 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 -- 172.21.15.146:0/1862502892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c159480 msgr2=0x7fad9c067340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 --2- 172.21.15.146:0/1862502892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c159480 0x7fad9c067340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad9c071780 tx=0x7fad8c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 -- 172.21.15.146:0/1862502892 shutdown_connections 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 --2- 172.21.15.146:0/1862502892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c159480 0x7fad9c067340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 --2- 172.21.15.146:0/1862502892 >> v2:172.21.15.161:3300/0 conn(0x7fad9c1590b0 0x7fad9c066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 -- 172.21.15.146:0/1862502892 >> 172.21.15.146:0/1862502892 conn(0x7fad9c0d0b90 msgr2=0x7fad9c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 -- 172.21.15.146:0/1862502892 shutdown_connections 2024-08-15T17:00:12.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.597+0000 7fad9b59e700 1 -- 172.21.15.146:0/1862502892 wait complete. 2024-08-15T17:00:12.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 Processor -- start 2024-08-15T17:00:12.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- start start 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad9c1590b0 0x7fad9c16d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c159480 0x7fad9c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c7dc0 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0b4780 con 0x7fad9c159480 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c7490 con 0x7fad9c1590b0 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55418/0 (socket says 172.21.15.146:55418) 2024-08-15T17:00:12.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 -- 172.21.15.146:0/2668866809 learned_addr learned my addr 172.21.15.146:0/2668866809 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:12.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad99d9b700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad9c1590b0 0x7fad9c16d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9ad9d700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c159480 0x7fad9c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 -- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad9c1590b0 msgr2=0x7fad9c16d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad9c1590b0 0x7fad9c16d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 -- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c159480 msgr2=0x7fad9c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c159480 0x7fad9c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 -- 172.21.15.146:0/2668866809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c009420 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9a59c700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad9000ec80 tx=0x7fad900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000c850 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9000c9b0 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c0c7770 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9000cd50 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.601+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c0aef80 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c064330 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fad90013680 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad8a7fc700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 0x7fad7c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fad9007cdd0 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad99d9b700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 0x7fad7c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.605+0000 7fad99d9b700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 0x7fad7c062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad9c071780 tx=0x7fad8c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.613+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad9004d220 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.777+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad9c0cf770 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.777+0000 7fad8a7fc700 1 -- 172.21.15.146:0/2668866809 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad90050840 con 0x7fad9c0cd2b0 2024-08-15T17:00:12.788 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:12.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 msgr2=0x7fad7c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 0x7fad7c062d10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad9c071780 tx=0x7fad8c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:12.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 msgr2=0x7fad9c16cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad9000ec80 tx=0x7fad900091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 shutdown_connections 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad7c060860 0x7fad7c062d10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad9c159480 0x7fad9c0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad9c1590b0 0x7fad9c16d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 --2- 172.21.15.146:0/2668866809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad9c0cd2b0 0x7fad9c16cdc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 >> 172.21.15.146:0/2668866809 conn(0x7fad9c0d0b90 msgr2=0x7fad9c063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 shutdown_connections 2024-08-15T17:00:12.793 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.781+0000 7fad9b59e700 1 -- 172.21.15.146:0/2668866809 wait complete. 2024-08-15T17:00:12.808 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:12.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 Processor -- start 2024-08-15T17:00:12.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 -- start start 2024-08-15T17:00:12.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00cca00 con 0x7fb7e00c6520 2024-08-15T17:00:12.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e015cf30 con 0x7fb7e015cb60 2024-08-15T17:00:12.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7e650b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e00b2e80 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.893+0000 7fb7d77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55422/0 (socket says 172.21.15.146:55422) 2024-08-15T17:00:12.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e4aa8700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb7e00c6520 0x7fb7e01631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39404/0 (socket says 172.21.15.146:39404) 2024-08-15T17:00:12.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 -- 172.21.15.146:0/2780080905 learned_addr learned my addr 172.21.15.146:0/2780080905 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 -- 172.21.15.146:0/2780080905 >> v1:172.21.15.161:6789/0 conn(0x7fb7e015cb60 legacy=0x7fb7e00cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 -- 172.21.15.146:0/2780080905 >> v1:172.21.15.146:6789/0 conn(0x7fb7e00c6520 legacy=0x7fb7e01631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 -- 172.21.15.146:0/2780080905 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00cbb50 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7df7fe700 1 -- 172.21.15.146:0/2780080905 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149459933 0 0) 0x7fb7e00cca00 con 0x7fb7e00c6520 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7d77fe700 1 --2- 172.21.15.146:0/2780080905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80099b0 tx=0x7fb7c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ce2a7acf5748a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7df7fe700 1 -- 172.21.15.146:0/2780080905 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800b860 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7df7fe700 1 -- 172.21.15.146:0/2780080905 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c800b9c0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7df7fe700 1 -- 172.21.15.146:0/2780080905 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800bd60 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 -- 172.21.15.146:0/2780080905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 msgr2=0x7fb7e00ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 --2- 172.21.15.146:0/2780080905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80099b0 tx=0x7fb7c8009250 comp rx=0 tx=0).stop 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 -- 172.21.15.146:0/2780080905 shutdown_connections 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 --2- 172.21.15.146:0/2780080905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00ceae0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 -- 172.21.15.146:0/2780080905 >> 172.21.15.146:0/2780080905 conn(0x7fb7e00996d0 msgr2=0x7fb7e00675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 -- 172.21.15.146:0/2780080905 shutdown_connections 2024-08-15T17:00:12.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.897+0000 7fb7e650b700 1 -- 172.21.15.146:0/2780080905 wait complete. 2024-08-15T17:00:12.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 Processor -- start 2024-08-15T17:00:12.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 -- start start 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7e00c6520 0x7fb7e015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7e015cb60 0x7fb7e015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e015bfc0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e016a7d0 con 0x7fb7e015cb60 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e650b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e015c3e0 con 0x7fb7e00c6520 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55436/0 (socket says 172.21.15.146:55436) 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 -- 172.21.15.146:0/1945495604 learned_addr learned my addr 172.21.15.146:0/1945495604 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7dffff700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7e00c6520 0x7fb7e015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7e4aa8700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7e015cb60 0x7fb7e015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 -- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7e00c6520 msgr2=0x7fb7e015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7e00c6520 0x7fb7e015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 -- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7e015cb60 msgr2=0x7fb7e015f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:12.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7e015cb60 0x7fb7e015f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:12.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.901+0000 7fb7d77fe700 1 -- 172.21.15.146:0/1945495604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c80094f0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7d77fe700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006f50 tx=0x7fb7c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c801f070 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c8011dc0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c801a690 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e0169bc0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e016a180 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.917 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e00645c0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.918 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb7c801aa00 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.905+0000 7fb7dcff9700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 0x7fb7c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:12.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.909+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb7c807e3e0 con 0x7fb7e00cd5a0 2024-08-15T17:00:12.919 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.909+0000 7fb7dffff700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 0x7fb7c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:12.920 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.909+0000 7fb7dffff700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 0x7fb7c4062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00cd0b0 tx=0x7fb7d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:12.925 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:12.913+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb7c804e8b0 con 0x7fb7e00cd5a0 2024-08-15T17:00:13.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7e016a910 con 0x7fb7e00cd5a0 2024-08-15T17:00:13.088 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7dcff9700 1 -- 172.21.15.146:0/1945495604 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7e016a910 con 0x7fb7e00cd5a0 2024-08-15T17:00:13.088 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:13.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 msgr2=0x7fb7c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 0x7fb7c4062d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00cd0b0 tx=0x7fb7d0009040 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 msgr2=0x7fb7e00c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8006f50 tx=0x7fb7c8017000 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 shutdown_connections 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7c4060860 0x7fb7c4062d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7e015cb60 0x7fb7e015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7e00cd5a0 0x7fb7e00c9310 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 --2- 172.21.15.146:0/1945495604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7e00c6520 0x7fb7e015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 >> 172.21.15.146:0/1945495604 conn(0x7fb7e00996d0 msgr2=0x7fb7e00695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 shutdown_connections 2024-08-15T17:00:13.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.077+0000 7fb7e650b700 1 -- 172.21.15.146:0/1945495604 wait complete. 2024-08-15T17:00:13.173 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-08-15T17:00:13.173 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:00:13.173 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:13.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.273+0000 7f008ebfc700 1 Processor -- start 2024-08-15T17:00:13.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.273+0000 7f008ebfc700 1 -- start start 2024-08-15T17:00:13.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 0x7f00880687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008815ce60 con 0x7f00880ae000 2024-08-15T17:00:13.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008815d0b0 con 0x7f008806da00 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008815d390 con 0x7f0088153a50 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f0087fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f00880ae000 0x7f008806d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39420/0 (socket says 172.21.15.146:39420) 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f0087fff700 1 -- 172.21.15.146:0/2572828264 learned_addr learned my addr 172.21.15.146:0/2572828264 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008c998700 1 --2- 172.21.15.146:0/2572828264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 0x7f00880687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578672616 0 0) 0x7f008815ce60 con 0x7f00880ae000 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0070003570 con 0x7f00880ae000 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277802302 0 0) 0x7f008815d0b0 con 0x7f008806da00 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f008815ce60 con 0x7f008806da00 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470344887 0 0) 0x7f0070003570 con 0x7f00880ae000 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008c998700 1 -- 172.21.15.146:0/2572828264 >> v1:172.21.15.161:6789/0 conn(0x7f008806da00 legacy=0x7f00880afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008c998700 1 -- 172.21.15.146:0/2572828264 >> v1:172.21.15.146:6789/0 conn(0x7f00880ae000 legacy=0x7f008806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008c998700 1 -- 172.21.15.146:0/2572828264 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008815d670 con 0x7f0088153a50 2024-08-15T17:00:13.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008c998700 1 --2- 172.21.15.146:0/2572828264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 0x7f00880687a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f007c009910 tx=0x7f007c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe4cc718cc440fb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0078001470 con 0x7f0088153a50 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c011420 con 0x7f0088153a50 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f00877fe700 1 -- 172.21.15.146:0/2572828264 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c0117c0 con 0x7f0088153a50 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 -- 172.21.15.146:0/2572828264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 msgr2=0x7f00880687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 --2- 172.21.15.146:0/2572828264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 0x7f00880687a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f007c009910 tx=0x7f007c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 -- 172.21.15.146:0/2572828264 shutdown_connections 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.277+0000 7f008ebfc700 1 --2- 172.21.15.146:0/2572828264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0088153a50 0x7f00880687a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- 172.21.15.146:0/2572828264 >> 172.21.15.146:0/2572828264 conn(0x7f00880ceb30 msgr2=0x7f008806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- 172.21.15.146:0/2572828264 shutdown_connections 2024-08-15T17:00:13.292 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- 172.21.15.146:0/2572828264 wait complete. 2024-08-15T17:00:13.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 Processor -- start 2024-08-15T17:00:13.293 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- start start 2024-08-15T17:00:13.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00880ae000 0x7f00880c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 0x7f00881497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.295 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880c7d90 con 0x7f008806da00 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880c8070 con 0x7f0088153a50 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008ebfc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00880c8350 con 0x7f00880ae000 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008c998700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008c998700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55452/0 (socket says 172.21.15.146:55452) 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008c998700 1 -- 172.21.15.146:0/226327694 learned_addr learned my addr 172.21.15.146:0/226327694 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f0087fff700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00880ae000 0x7f00880c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.281+0000 7f008d199700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 0x7f00881497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 -- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00880ae000 msgr2=0x7f00880c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00880ae000 0x7f00880c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 -- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 msgr2=0x7f00881497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 0x7f00881497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 -- 172.21.15.146:0/226327694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c009420 con 0x7f008806da00 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008d199700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 0x7f00881497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:13.296 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008c998700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f007c0091e0 tx=0x7f007c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c01cbe0 con 0x7f008806da00 2024-08-15T17:00:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007c01cd40 con 0x7f008806da00 2024-08-15T17:00:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00880c8630 con 0x7f008806da00 2024-08-15T17:00:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007c01a6a0 con 0x7f008806da00 2024-08-15T17:00:13.297 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0088156de0 con 0x7f008806da00 2024-08-15T17:00:13.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00880ca970 con 0x7f008806da00 2024-08-15T17:00:13.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f007c01aad0 con 0x7f008806da00 2024-08-15T17:00:13.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.285+0000 7f0084ff9700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 0x7f0074062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.289+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f007c07ea30 con 0x7f008806da00 2024-08-15T17:00:13.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.289+0000 7f0087fff700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 0x7f0074062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.289+0000 7f0087fff700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 0x7f0074062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00880cd230 tx=0x7f0078000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.293+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f00880ca970 con 0x7f008806da00 2024-08-15T17:00:13.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.457+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00880ca970 con 0x7f008806da00 2024-08-15T17:00:13.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.457+0000 7f0084ff9700 1 -- 172.21.15.146:0/226327694 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f00880ca970 con 0x7f008806da00 2024-08-15T17:00:13.470 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:13.470 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:13.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 msgr2=0x7f0074062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 0x7f0074062d10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00880cd230 tx=0x7f0078000bc0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 msgr2=0x7f00880b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f007c0091e0 tx=0x7f007c016000 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 shutdown_connections 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0074060860 0x7f0074062d10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0088153a50 0x7f00881497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00880ae000 0x7f00880c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 --2- 172.21.15.146:0/226327694 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f008806da00 0x7f00880b1250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 >> 172.21.15.146:0/226327694 conn(0x7f00880ceb30 msgr2=0x7f00880ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 shutdown_connections 2024-08-15T17:00:13.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.461+0000 7f008ebfc700 1 -- 172.21.15.146:0/226327694 wait complete. 2024-08-15T17:00:13.487 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:13.488 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:13.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 Processor -- start 2024-08-15T17:00:13.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 -- start start 2024-08-15T17:00:13.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc0701538e0 0x7fc0700b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0700b0ad0 con 0x7fc07006f350 2024-08-15T17:00:13.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07015a940 con 0x7fc070153cb0 2024-08-15T17:00:13.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc075ebe700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0700b01a0 con 0x7fc0701538e0 2024-08-15T17:00:13.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc06ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc07006f350 0x7fc070159bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39434/0 (socket says 172.21.15.146:39434) 2024-08-15T17:00:13.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc06ffff700 1 -- 172.21.15.146:0/2978915632 learned_addr learned my addr 172.21.15.146:0/2978915632 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:13.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc06f7fe700 1 --2- 172.21.15.146:0/2978915632 >> v2:172.21.15.161:3300/0 conn(0x7fc0701538e0 0x7fc0700b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.585+0000 7fc06e7fc700 1 -- 172.21.15.146:0/2978915632 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258733354 0 0) 0x7fc0700b0ad0 con 0x7fc07006f350 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06e7fc700 1 -- 172.21.15.146:0/2978915632 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc058003570 con 0x7fc07006f350 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06f7fe700 1 -- 172.21.15.146:0/2978915632 >> v1:172.21.15.161:6789/0 conn(0x7fc070153cb0 legacy=0x7fc07006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06f7fe700 1 -- 172.21.15.146:0/2978915632 >> v1:172.21.15.146:6789/0 conn(0x7fc07006f350 legacy=0x7fc070159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06f7fe700 1 -- 172.21.15.146:0/2978915632 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0700b0480 con 0x7fc0701538e0 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06f7fe700 1 --2- 172.21.15.146:0/2978915632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0701538e0 0x7fc0700b0f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0640098e0 tx=0x7fc0640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e84250451261cf server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06e7fc700 1 -- 172.21.15.146:0/2978915632 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400b640 con 0x7fc0701538e0 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06e7fc700 1 -- 172.21.15.146:0/2978915632 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400b7a0 con 0x7fc0701538e0 2024-08-15T17:00:13.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc06e7fc700 1 -- 172.21.15.146:0/2978915632 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400bb40 con 0x7fc0701538e0 2024-08-15T17:00:13.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- 172.21.15.146:0/2978915632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0701538e0 msgr2=0x7fc0700b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 --2- 172.21.15.146:0/2978915632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0701538e0 0x7fc0700b0f60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc0640098e0 tx=0x7fc0640091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- 172.21.15.146:0/2978915632 shutdown_connections 2024-08-15T17:00:13.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 --2- 172.21.15.146:0/2978915632 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc0701538e0 0x7fc0700b0f60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- 172.21.15.146:0/2978915632 >> 172.21.15.146:0/2978915632 conn(0x7fc070066e00 msgr2=0x7fc070067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- 172.21.15.146:0/2978915632 shutdown_connections 2024-08-15T17:00:13.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- 172.21.15.146:0/2978915632 wait complete. 2024-08-15T17:00:13.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 Processor -- start 2024-08-15T17:00:13.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.589+0000 7fc075ebe700 1 -- start start 2024-08-15T17:00:13.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc07006f350 0x7fc07014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 0x7fc070097670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07014a200 con 0x7fc070153cb0 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc070158440 con 0x7fc07006f350 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc070158f90 con 0x7fc0701538e0 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55458/0 (socket says 172.21.15.146:55458) 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 -- 172.21.15.146:0/1379468482 learned_addr learned my addr 172.21.15.146:0/1379468482 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06f7fe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc07006f350 0x7fc07014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06effd700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 0x7fc070097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 -- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 msgr2=0x7fc070097670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 0x7fc070097670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 -- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc07006f350 msgr2=0x7fc07014a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc07006f350 0x7fc07014a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 -- 172.21.15.146:0/1379468482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064009420 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06effd700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 0x7fc070097670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc06ffff700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc06800cc40 tx=0x7fc068001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06800a910 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06800aa70 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0700cb770 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06800adf0 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.593+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc070158940 con 0x7fc070153cb0 2024-08-15T17:00:13.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc070097400 con 0x7fc070153cb0 2024-08-15T17:00:13.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc068007300 con 0x7fc070153cb0 2024-08-15T17:00:13.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc0577fe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 0x7fc05c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc06807cf80 con 0x7fc070153cb0 2024-08-15T17:00:13.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc06f7fe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 0x7fc05c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.597+0000 7fc06f7fe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 0x7fc05c062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0640093f0 tx=0x7fc064017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.605+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc06804d450 con 0x7fc070153cb0 2024-08-15T17:00:13.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.765+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0700645c0 con 0x7fc070153cb0 2024-08-15T17:00:13.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc0577fe700 1 -- 172.21.15.146:0/1379468482 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc068050a70 con 0x7fc070153cb0 2024-08-15T17:00:13.779 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 msgr2=0x7fc05c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 0x7fc05c062d10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc0640093f0 tx=0x7fc064017000 comp rx=0 tx=0).stop 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 msgr2=0x7fc070097bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc06800cc40 tx=0x7fc068001aa0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 shutdown_connections 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc05c060860 0x7fc05c062d10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc070153cb0 0x7fc070097bb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0701538e0 0x7fc070097670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 --2- 172.21.15.146:0/1379468482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc07006f350 0x7fc07014a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 >> 172.21.15.146:0/1379468482 conn(0x7fc070066e00 msgr2=0x7fc0700694b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 shutdown_connections 2024-08-15T17:00:13.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.769+0000 7fc075ebe700 1 -- 172.21.15.146:0/1379468482 wait complete. 2024-08-15T17:00:13.794 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:13.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.889+0000 7f236b59e700 1 Processor -- start 2024-08-15T17:00:13.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.889+0000 7f236b59e700 1 -- start start 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f236c0cbcc0 0x7f236c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0cc350 con 0x7f236c0b0390 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0ca760 con 0x7f236c0cbcc0 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0cd3f0 con 0x7f236c15d9f0 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f236c0cbcc0 0x7f236c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:35052/0 (socket says 172.21.15.146:35052) 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236ad9d700 1 -- 172.21.15.146:0/431708343 learned_addr learned my addr 172.21.15.146:0/431708343 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:13.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236a59c700 1 --1- 172.21.15.146:0/431708343 >> v1:172.21.15.161:6789/0 conn(0x7f236c0b0390 0x7f236c15cda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:39314/0 (socket says 172.21.15.146:39314) 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 -- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 msgr2=0x7f236c0c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 -- 172.21.15.146:0/431708343 >> v1:172.21.15.161:6789/0 conn(0x7f236c0b0390 legacy=0x7f236c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236959a700 1 -- 172.21.15.146:0/431708343 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2774598171 0 0) 0x7f236c0cc350 con 0x7f236c0b0390 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 -- 172.21.15.146:0/431708343 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0cc630 con 0x7f236c0cbcc0 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236ad9d700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:13.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f2369d9b700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c15d4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f23540098e0 tx=0x7f23540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef476d0a37756a77 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236959a700 1 -- 172.21.15.146:0/431708343 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400b640 con 0x7f236c0cbcc0 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236959a700 1 -- 172.21.15.146:0/431708343 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235400b7a0 con 0x7f236c0cbcc0 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236959a700 1 -- 172.21.15.146:0/431708343 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400bb40 con 0x7f236c0cbcc0 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 msgr2=0x7f236c15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c15d4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f23540098e0 tx=0x7f23540091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- 172.21.15.146:0/431708343 shutdown_connections 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c15d9f0 0x7f236c0c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 --2- 172.21.15.146:0/431708343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c15d4b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- 172.21.15.146:0/431708343 >> 172.21.15.146:0/431708343 conn(0x7f236c0cdee0 msgr2=0x7f236c0ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- 172.21.15.146:0/431708343 shutdown_connections 2024-08-15T17:00:13.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.893+0000 7f236b59e700 1 -- 172.21.15.146:0/431708343 wait complete. 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 Processor -- start 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- start start 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c0b0390 0x7f236c15f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c16cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c154b80 con 0x7f236c15d9f0 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15ed90 con 0x7f236c0cbcc0 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0c7f30 con 0x7f236c0b0390 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55470/0 (socket says 172.21.15.146:55470) 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 -- 172.21.15.146:0/108975557 learned_addr learned my addr 172.21.15.146:0/108975557 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f2369d9b700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c16cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236a59c700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c0b0390 0x7f236c15f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 -- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c0b0390 msgr2=0x7f236c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c0b0390 0x7f236c15f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 -- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 msgr2=0x7f236c16cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c16cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:13.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 -- 172.21.15.146:0/108975557 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009ae0 con 0x7f236c15d9f0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f2369d9b700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c16cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236ad9d700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2364009f40 tx=0x7f236400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400c960 con 0x7f236c15d9f0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400f040 con 0x7f236c15d9f0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400cd30 con 0x7f236c15d9f0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2354009420 con 0x7f236c15d9f0 2024-08-15T17:00:13.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.897+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c15ea20 con 0x7f236c15d9f0 2024-08-15T17:00:13.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c15e890 con 0x7f236c15d9f0 2024-08-15T17:00:13.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2364014690 con 0x7f236c15d9f0 2024-08-15T17:00:13.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f23627fc700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 0x7f2350062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:13.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f236407d1a0 con 0x7f236c15d9f0 2024-08-15T17:00:13.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f236a59c700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 0x7f2350062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:13.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.901+0000 7f236a59c700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 0x7f2350062d10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f235c007fd0 tx=0x7f235c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:13.922 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:13.909+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f236404d5f0 con 0x7f236c15d9f0 2024-08-15T17:00:14.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.073+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f236c0638f0 con 0x7f236c15d9f0 2024-08-15T17:00:14.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.073+0000 7f23627fc700 1 -- 172.21.15.146:0/108975557 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2364050c10 con 0x7f236c15d9f0 2024-08-15T17:00:14.086 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 msgr2=0x7f2350062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 0x7f2350062d10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f235c007fd0 tx=0x7f235c007d50 comp rx=0 tx=0).stop 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 msgr2=0x7f236c16d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2364009f40 tx=0x7f236400b040 comp rx=0 tx=0).stop 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 shutdown_connections 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2350060860 0x7f2350062d10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f236c15d9f0 0x7f236c16d2a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f236c0cbcc0 0x7f236c16cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 --2- 172.21.15.146:0/108975557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f236c0b0390 0x7f236c15f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 >> 172.21.15.146:0/108975557 conn(0x7f236c0cdee0 msgr2=0x7f236c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 shutdown_connections 2024-08-15T17:00:14.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.077+0000 7f236b59e700 1 -- 172.21.15.146:0/108975557 wait complete. 2024-08-15T17:00:14.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:14.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 Processor -- start 2024-08-15T17:00:14.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 -- start start 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a2c153690 0x7f3a2c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3a2c153a60 0x7f3a2c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c15a700 con 0x7f3a2c06c740 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c1598e0 con 0x7f3a2c153690 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a31766700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c06cb50 con 0x7f3a2c153a60 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a2bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a2c153690 0x7f3a2c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a2bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3a2c153690 0x7f3a2c0b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55486/0 (socket says 172.21.15.146:55486) 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.201+0000 7f3a2bfff700 1 -- 172.21.15.146:0/2310342179 learned_addr learned my addr 172.21.15.146:0/2310342179 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:14.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2b7fe700 1 --2- 172.21.15.146:0/2310342179 >> v2:172.21.15.161:3300/0 conn(0x7f3a2c153a60 0x7f3a2c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2bfff700 1 -- 172.21.15.146:0/2310342179 >> v2:172.21.15.161:3300/0 conn(0x7f3a2c153a60 msgr2=0x7f3a2c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2bfff700 1 --2- 172.21.15.146:0/2310342179 >> v2:172.21.15.161:3300/0 conn(0x7f3a2c153a60 0x7f3a2c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2bfff700 1 -- 172.21.15.146:0/2310342179 >> v1:172.21.15.161:6789/0 conn(0x7f3a2c06c740 legacy=0x7f3a2c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2bfff700 1 -- 172.21.15.146:0/2310342179 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0b0f80 con 0x7f3a2c153690 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2bfff700 1 --2- 172.21.15.146:0/2310342179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153690 0x7f3a2c0b0a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06f830 tx=0x7f3a20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a712d696f0e4fe server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2affd700 1 -- 172.21.15.146:0/2310342179 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000b860 con 0x7f3a2c153690 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2affd700 1 -- 172.21.15.146:0/2310342179 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b9c0 con 0x7f3a2c153690 2024-08-15T17:00:14.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a2affd700 1 -- 172.21.15.146:0/2310342179 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000bd60 con 0x7f3a2c153690 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 -- 172.21.15.146:0/2310342179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153690 msgr2=0x7f3a2c0b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 --2- 172.21.15.146:0/2310342179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153690 0x7f3a2c0b0a40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06f830 tx=0x7f3a20009250 comp rx=0 tx=0).stop 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 -- 172.21.15.146:0/2310342179 shutdown_connections 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 --2- 172.21.15.146:0/2310342179 >> v2:172.21.15.161:3300/0 conn(0x7f3a2c153a60 0x7f3a2c06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 --2- 172.21.15.146:0/2310342179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153690 0x7f3a2c0b0a40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 -- 172.21.15.146:0/2310342179 >> 172.21.15.146:0/2310342179 conn(0x7f3a2c0ca2b0 msgr2=0x7f3a2c06baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 -- 172.21.15.146:0/2310342179 shutdown_connections 2024-08-15T17:00:14.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.205+0000 7f3a31766700 1 -- 172.21.15.146:0/2310342179 wait complete. 2024-08-15T17:00:14.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 Processor -- start 2024-08-15T17:00:14.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 -- start start 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a2c06c740 0x7f3a2c1503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a2c153690 0x7f3a2c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c154940 con 0x7f3a2c153a60 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c154050 con 0x7f3a2c153690 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a31766700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c155ee0 con 0x7f3a2c06c740 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a2b7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a2c153690 0x7f3a2c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a2bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a2c06c740 0x7f3a2c1503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55490/0 (socket says 172.21.15.146:55490) 2024-08-15T17:00:14.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 -- 172.21.15.146:0/2218048867 learned_addr learned my addr 172.21.15.146:0/2218048867 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:14.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 -- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a2c06c740 msgr2=0x7f3a2c1503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a2c06c740 0x7f3a2c1503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 -- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a2c153690 msgr2=0x7f3a2c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a2c153690 0x7f3a2c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.224 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.209+0000 7f3a30f65700 1 -- 172.21.15.146:0/2218048867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200094f0 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a30f65700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012630 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24012790 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c1542f0 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24012b50 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c1508f0 con 0x7f3a2c153a60 2024-08-15T17:00:14.225 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c154ab0 con 0x7f3a2c153a60 2024-08-15T17:00:14.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3a2401a3a0 con 0x7f3a2c153a60 2024-08-15T17:00:14.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.213+0000 7f3a13fff700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 0x7f3a18062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.217+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a2407cde0 con 0x7f3a2c153a60 2024-08-15T17:00:14.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.217+0000 7f3a2bfff700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 0x7f3a18062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.217+0000 7f3a2bfff700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 0x7f3a18062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06f830 tx=0x7f3a20017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.225+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3a24050c20 con 0x7f3a2c153a60 2024-08-15T17:00:14.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a2c0638f0 con 0x7f3a2c153a60 2024-08-15T17:00:14.396 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a13fff700 1 -- 172.21.15.146:0/2218048867 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a240507b0 con 0x7f3a2c153a60 2024-08-15T17:00:14.396 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 msgr2=0x7f3a18062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 0x7f3a18062d10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c06f830 tx=0x7f3a20017000 comp rx=0 tx=0).stop 2024-08-15T17:00:14.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 msgr2=0x7f3a2c0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c8e0 tx=0x7f3a24001710 comp rx=0 tx=0).stop 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 shutdown_connections 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3a18060860 0x7f3a18062d10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a2c153a60 0x7f3a2c0cbf30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3a2c153690 0x7f3a2c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 --2- 172.21.15.146:0/2218048867 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3a2c06c740 0x7f3a2c1503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 >> 172.21.15.146:0/2218048867 conn(0x7f3a2c0ca2b0 msgr2=0x7f3a2c15aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 shutdown_connections 2024-08-15T17:00:14.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.385+0000 7f3a31766700 1 -- 172.21.15.146:0/2218048867 wait complete. 2024-08-15T17:00:14.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T17:00:14.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 Processor -- start 2024-08-15T17:00:14.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 -- start start 2024-08-15T17:00:14.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f4015b6b0 0x7f9f400b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 0x7f9f40158a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4015ac60 con 0x7f9f40140900 2024-08-15T17:00:14.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400b2d60 con 0x7f9f4015b6b0 2024-08-15T17:00:14.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.501+0000 7f9f44bd2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4015b220 con 0x7f9f400b28e0 2024-08-15T17:00:14.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9f4015b6b0 0x7f9f400b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3e59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f9f40140900 0x7f9f40157680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39444/0 (socket says 172.21.15.146:39444) 2024-08-15T17:00:14.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3e59c700 1 -- 172.21.15.146:0/1196518552 learned_addr learned my addr 172.21.15.146:0/1196518552 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:14.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3ed9d700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 0x7f9f40158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 -- 172.21.15.146:0/1196518552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 msgr2=0x7f9f40158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 0x7f9f40158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 -- 172.21.15.146:0/1196518552 >> v1:172.21.15.146:6789/0 conn(0x7f9f40140900 legacy=0x7f9f40157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 -- 172.21.15.146:0/1196518552 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f401581b0 con 0x7f9f4015b6b0 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3d59a700 1 -- 172.21.15.146:0/1196518552 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591338886 0 0) 0x7f9f4015ac60 con 0x7f9f40140900 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3ed9d700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 0x7f9f40158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3dd9b700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f400b23a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098e0 tx=0x7f9f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac9c14f7c3ff9cc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3d59a700 1 -- 172.21.15.146:0/1196518552 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800b640 con 0x7f9f4015b6b0 2024-08-15T17:00:14.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3d59a700 1 -- 172.21.15.146:0/1196518552 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2800b7a0 con 0x7f9f4015b6b0 2024-08-15T17:00:14.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f3d59a700 1 -- 172.21.15.146:0/1196518552 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800bb40 con 0x7f9f4015b6b0 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1196518552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 msgr2=0x7f9f400b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f400b23a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9f280098e0 tx=0x7f9f280091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1196518552 shutdown_connections 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f400b28e0 0x7f9f40158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1196518552 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f400b23a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1196518552 >> 172.21.15.146:0/1196518552 conn(0x7f9f4006e3b0 msgr2=0x7f9f400aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1196518552 shutdown_connections 2024-08-15T17:00:14.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.505+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1196518552 wait complete. 2024-08-15T17:00:14.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 Processor -- start 2024-08-15T17:00:14.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 -- start start 2024-08-15T17:00:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f400b28e0 0x7f9f40168430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f40140900 0x7f9f400c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c31e0 con 0x7f9f4015b6b0 2024-08-15T17:00:14.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c25d0 con 0x7f9f400b28e0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f44bd2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c28b0 con 0x7f9f40140900 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f3ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f3ed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55504/0 (socket says 172.21.15.146:55504) 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.509+0000 7f9f3ed9d700 1 -- 172.21.15.146:0/1566543478 learned_addr learned my addr 172.21.15.146:0/1566543478 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3e59c700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f400b28e0 0x7f9f40168430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3dd9b700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f40140900 0x7f9f400c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 -- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f40140900 msgr2=0x7f9f400c3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f40140900 0x7f9f400c3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 -- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f400b28e0 msgr2=0x7f9f40168430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f400b28e0 0x7f9f40168430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 -- 172.21.15.146:0/1566543478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009a20 con 0x7f9f4015b6b0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f3ed9d700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009f20 tx=0x7f9f3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800cb70 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3800f040 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38014480 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f28009420 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400c2e70 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f400c3350 con 0x7f9f4015b6b0 2024-08-15T17:00:14.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.513+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9f38003020 con 0x7f9f4015b6b0 2024-08-15T17:00:14.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.517+0000 7f9f367fc700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 0x7f9f20062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.517+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f3807d4e0 con 0x7f9f4015b6b0 2024-08-15T17:00:14.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.517+0000 7f9f3e59c700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 0x7f9f20062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.517+0000 7f9f3e59c700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 0x7f9f20062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007fd0 tx=0x7f9f30007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.521+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9f3804d930 con 0x7f9f4015b6b0 2024-08-15T17:00:14.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.741+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f9f400645c0 con 0x7f9f4015b6b0 2024-08-15T17:00:14.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f367fc700 1 -- 172.21.15.146:0/1566543478 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9f38050f50 con 0x7f9f4015b6b0 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 msgr2=0x7f9f20062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 0x7f9f20062d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007fd0 tx=0x7f9f30007d30 comp rx=0 tx=0).stop 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 msgr2=0x7f9f40167220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009f20 tx=0x7f9f3800b040 comp rx=0 tx=0).stop 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 shutdown_connections 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9f20060860 0x7f9f20062d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f4015b6b0 0x7f9f40167220 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f40140900 0x7f9f400c3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 --2- 172.21.15.146:0/1566543478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f400b28e0 0x7f9f40168430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 >> 172.21.15.146:0/1566543478 conn(0x7f9f4006e3b0 msgr2=0x7f9f400aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.817+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 shutdown_connections 2024-08-15T17:00:14.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.821+0000 7f9f44bd2700 1 -- 172.21.15.146:0/1566543478 wait complete. 2024-08-15T17:00:14.850 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:00:14.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 Processor -- start 2024-08-15T17:00:14.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 -- start start 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed8067030 0x7f2ed8067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed80cf670 con 0x7f2ed80cb5c0 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80cea60 con 0x7f2ed8067030 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.961+0000 7f2edcfb6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80ced40 con 0x7f2ed80cfb00 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed8067030 0x7f2ed8067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2ed8067030 0x7f2ed8067400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55512/0 (socket says 172.21.15.146:55512) 2024-08-15T17:00:14.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3465940936 learned_addr learned my addr 172.21.15.146:0/3465940936 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:14.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed6ffd700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3465940936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 msgr2=0x7f2ed80cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed80cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3465940936 >> v1:172.21.15.161:6789/0 conn(0x7f2ed80cb5c0 legacy=0x7f2ed8158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3465940936 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed80cf020 con 0x7f2ed8067030 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed8067400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80099b0 tx=0x7f2ec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5693660ef3d68cbd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed67fc700 1 -- 172.21.15.146:0/3465940936 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800b860 con 0x7f2ed8067030 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed67fc700 1 -- 172.21.15.146:0/3465940936 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec800b9c0 con 0x7f2ed8067030 2024-08-15T17:00:14.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2ed67fc700 1 -- 172.21.15.146:0/3465940936 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800bd60 con 0x7f2ed8067030 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3465940936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 msgr2=0x7f2ed8067400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed8067400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80099b0 tx=0x7f2ec8009250 comp rx=0 tx=0).stop 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3465940936 shutdown_connections 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3465940936 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed8067400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3465940936 >> 172.21.15.146:0/3465940936 conn(0x7f2ed80d0b50 msgr2=0x7f2ed80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3465940936 shutdown_connections 2024-08-15T17:00:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.965+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3465940936 wait complete. 2024-08-15T17:00:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 Processor -- start 2024-08-15T17:00:14.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- start start 2024-08-15T17:00:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ed80cb5c0 0x7f2ed8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed816a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c7e20 con 0x7f2ed8067030 2024-08-15T17:00:14.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c7210 con 0x7f2ed80cb5c0 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80c74f0 con 0x7f2ed80cfb00 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55528/0 (socket says 172.21.15.146:55528) 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3506440833 learned_addr learned my addr 172.21.15.146:0/3506440833 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed6ffd700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ed80cb5c0 0x7f2ed8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed7fff700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed816a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 msgr2=0x7f2ed816a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed816a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ed80cb5c0 msgr2=0x7f2ed8158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ed80cb5c0 0x7f2ed8158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 -- 172.21.15.146:0/3506440833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80094f0 con 0x7f2ed8067030 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ed77fe700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8069520 tx=0x7f2ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8024440 con 0x7f2ed8067030 2024-08-15T17:00:14.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec80245a0 con 0x7f2ed8067030 2024-08-15T17:00:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed80c77d0 con 0x7f2ed8067030 2024-08-15T17:00:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8024960 con 0x7f2ed8067030 2024-08-15T17:00:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.969+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed815c5e0 con 0x7f2ed8067030 2024-08-15T17:00:14.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed80638f0 con 0x7f2ed8067030 2024-08-15T17:00:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2ec801a420 con 0x7f2ed8067030 2024-08-15T17:00:14.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2ebf7fe700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 0x7f2ec4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2ec807e960 con 0x7f2ed8067030 2024-08-15T17:00:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2ed6ffd700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 0x7f2ec4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:14.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.973+0000 7f2ed6ffd700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 0x7f2ec4062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cb130 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:14.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:14.981+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2ec804edb0 con 0x7f2ed8067030 2024-08-15T17:00:15.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.149+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2ed80c5b10 con 0x7f2ed8067030 2024-08-15T17:00:15.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.153+0000 7f2ebf7fe700 1 -- 172.21.15.146:0/3506440833 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ec80150c0 con 0x7f2ed8067030 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 msgr2=0x7f2ec4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 0x7f2ec4062d10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cb130 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 msgr2=0x7f2ed80c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8069520 tx=0x7f2ec8017000 comp rx=0 tx=0).stop 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 shutdown_connections 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2ec4060860 0x7f2ec4062d10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ed80cfb00 0x7f2ed816a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ed80cb5c0 0x7f2ed8158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 --2- 172.21.15.146:0/3506440833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed8067030 0x7f2ed80c82b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 >> 172.21.15.146:0/3506440833 conn(0x7f2ed80d0b50 msgr2=0x7f2ed80b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 shutdown_connections 2024-08-15T17:00:15.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.157+0000 7f2edcfb6700 1 -- 172.21.15.146:0/3506440833 wait complete. 2024-08-15T17:00:15.181 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T17:00:15.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:00:15.199 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:00:15.220 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T17:00:15.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-08-15T17:00:15.221 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:00:15.221 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:15.273 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:00:15.273 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:00:15.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:15.273 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T17:00:15.300 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi161.front.sepia.ceph.com... 2024-08-15T17:00:15.300 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:00:15.300 DEBUG:teuthology.orchestra.run.smithi161:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:15.332 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:00:15.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:00:15.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:15.332 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T17:00:15.403 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-08-15T17:00:15.403 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:00:15.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-08-15T17:00:15.421 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:00:15.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:00:15.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:15.421 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T17:00:15.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi161.front.sepia.ceph.com... 2024-08-15T17:00:15.471 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:00:15.471 DEBUG:teuthology.orchestra.run.smithi161:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-08-15T17:00:15.487 INFO:teuthology.orchestra.run:waiting for 900 2024-08-15T17:00:15.487 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:00:15.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:15.487 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T17:00:15.492 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-08-15T17:00:15.492 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:15.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 Processor -- start 2024-08-15T17:00:15.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 -- start start 2024-08-15T17:00:15.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdb3c156f80 0x7fdb3c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb3c06f040 0x7fdb3c0aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3c0adf50 con 0x7fdb3c0ace80 2024-08-15T17:00:15.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ae920 con 0x7fdb3c06f040 2024-08-15T17:00:15.607 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb4393a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ad340 con 0x7fdb3c156f80 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb41ed7700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdb3c06f040 0x7fdb3c0aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb40ed5700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdb3c0ace80 0x7fdb3c157350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39460/0 (socket says 172.21.15.146:39460) 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.593+0000 7fdb40ed5700 1 -- 172.21.15.146:0/745174570 learned_addr learned my addr 172.21.15.146:0/745174570 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb416d6700 1 --2- 172.21.15.146:0/745174570 >> v2:172.21.15.161:3300/0 conn(0x7fdb3c156f80 0x7fdb3c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb41ed7700 1 -- 172.21.15.146:0/745174570 >> v2:172.21.15.161:3300/0 conn(0x7fdb3c156f80 msgr2=0x7fdb3c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb41ed7700 1 --2- 172.21.15.146:0/745174570 >> v2:172.21.15.161:3300/0 conn(0x7fdb3c156f80 0x7fdb3c0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb41ed7700 1 -- 172.21.15.146:0/745174570 >> v1:172.21.15.146:6789/0 conn(0x7fdb3c0ace80 legacy=0x7fdb3c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb41ed7700 1 -- 172.21.15.146:0/745174570 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c0ad620 con 0x7fdb3c06f040 2024-08-15T17:00:15.608 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb41ed7700 1 --2- 172.21.15.146:0/745174570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c0aab70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c06c3b0 tx=0x7fdb340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5a0a829b6880fd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb33fff700 1 -- 172.21.15.146:0/745174570 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400b6b0 con 0x7fdb3c06f040 2024-08-15T17:00:15.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb33fff700 1 -- 172.21.15.146:0/745174570 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3400b810 con 0x7fdb3c06f040 2024-08-15T17:00:15.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb33fff700 1 -- 172.21.15.146:0/745174570 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400bbb0 con 0x7fdb3c06f040 2024-08-15T17:00:15.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 -- 172.21.15.146:0/745174570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 msgr2=0x7fdb3c0aab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 --2- 172.21.15.146:0/745174570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c0aab70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c06c3b0 tx=0x7fdb340091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 -- 172.21.15.146:0/745174570 shutdown_connections 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 --2- 172.21.15.146:0/745174570 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c0aab70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 --2- 172.21.15.146:0/745174570 >> v2:172.21.15.161:3300/0 conn(0x7fdb3c156f80 0x7fdb3c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 -- 172.21.15.146:0/745174570 >> 172.21.15.146:0/745174570 conn(0x7fdb3c066e00 msgr2=0x7fdb3c067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 -- 172.21.15.146:0/745174570 shutdown_connections 2024-08-15T17:00:15.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.597+0000 7fdb4393a700 1 -- 172.21.15.146:0/745174570 wait complete. 2024-08-15T17:00:15.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 Processor -- start 2024-08-15T17:00:15.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 -- start start 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb3c0ace80 0x7fdb3c14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb3c156f80 0x7fdb3c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c14f4d0 con 0x7fdb3c06f040 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c14ef10 con 0x7fdb3c0ace80 2024-08-15T17:00:15.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb4393a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c14f1f0 con 0x7fdb3c156f80 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb416d6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb416d6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55548/0 (socket says 172.21.15.146:55548) 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb416d6700 1 -- 172.21.15.146:0/2248124872 learned_addr learned my addr 172.21.15.146:0/2248124872 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb40ed5700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb3c0ace80 0x7fdb3c14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.601+0000 7fdb41ed7700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb3c156f80 0x7fdb3c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 -- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb3c156f80 msgr2=0x7fdb3c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb3c156f80 0x7fdb3c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 -- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb3c0ace80 msgr2=0x7fdb3c14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb3c0ace80 0x7fdb3c14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 -- 172.21.15.146:0/2248124872 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34009420 con 0x7fdb3c06f040 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb416d6700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d890 tx=0x7fdb2c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0125f0 con 0x7fdb3c06f040 2024-08-15T17:00:15.617 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c012750 con 0x7fdb3c06f040 2024-08-15T17:00:15.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c096b20 con 0x7fdb3c06f040 2024-08-15T17:00:15.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c012af0 con 0x7fdb3c06f040 2024-08-15T17:00:15.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c14c310 con 0x7fdb3c06f040 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdb2c01e460 con 0x7fdb3c06f040 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c096fa0 con 0x7fdb3c06f040 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 0x7fdb20062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdb2c07d170 con 0x7fdb3c06f040 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.605+0000 7fdb40ed5700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 0x7fdb20062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.609+0000 7fdb40ed5700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 0x7fdb20062de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1569c0 tx=0x7fdb38001a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.617+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdb2c051030 con 0x7fdb3c06f040 2024-08-15T17:00:15.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.821+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb3c0645c0 con 0x7fdb3c06f040 2024-08-15T17:00:15.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.821+0000 7fdb317fa700 1 -- 172.21.15.146:0/2248124872 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fdb2c050bc0 con 0x7fdb3c06f040 2024-08-15T17:00:15.834 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:15.834 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:00:15.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 msgr2=0x7fdb20062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 0x7fdb20062de0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1569c0 tx=0x7fdb38001a10 comp rx=0 tx=0).stop 2024-08-15T17:00:15.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 msgr2=0x7fdb3c13ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d890 tx=0x7fdb2c008040 comp rx=0 tx=0).stop 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 shutdown_connections 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdb20060930 0x7fdb20062de0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb3c156f80 0x7fdb3c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb3c0ace80 0x7fdb3c14e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 --2- 172.21.15.146:0/2248124872 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb3c06f040 0x7fdb3c13ccc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 >> 172.21.15.146:0/2248124872 conn(0x7fdb3c066e00 msgr2=0x7fdb3c0a9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 shutdown_connections 2024-08-15T17:00:15.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.825+0000 7fdb4393a700 1 -- 172.21.15.146:0/2248124872 wait complete. 2024-08-15T17:00:15.838 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T17:00:15.847 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:15.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.941+0000 7fad4b59e700 1 Processor -- start 2024-08-15T17:00:15.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.941+0000 7fad4b59e700 1 -- start start 2024-08-15T17:00:15.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fad4c159280 0x7fad4c06e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c0ac000 con 0x7fad4c158eb0 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4c0ab190 con 0x7fad4c159280 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0ad4a0 con 0x7fad4c0aecc0 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fad4c158eb0 0x7fad4c0aaa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39470/0 (socket says 172.21.15.146:39470) 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/4144352811 >> v2:172.21.15.146:3300/0 conn(0x7fad4c159280 0x7fad4c06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4a59c700 1 -- 172.21.15.146:0/4144352811 learned_addr learned my addr 172.21.15.146:0/4144352811 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad49d9b700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 -- 172.21.15.146:0/4144352811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 msgr2=0x7fad4c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 -- 172.21.15.146:0/4144352811 >> v1:172.21.15.146:6789/0 conn(0x7fad4c158eb0 legacy=0x7fad4c0aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 -- 172.21.15.146:0/4144352811 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0ab3f0 con 0x7fad4c159280 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad49d9b700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c159280 0x7fad4c06e450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad440098e0 tx=0x7fad440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e5b42989bb801 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4959a700 1 -- 172.21.15.146:0/4144352811 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44009e80 con 0x7fad4c159280 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4959a700 1 -- 172.21.15.146:0/4144352811 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400b860 con 0x7fad4c159280 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4959a700 1 -- 172.21.15.146:0/4144352811 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400bbe0 con 0x7fad4c159280 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- 172.21.15.146:0/4144352811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c159280 msgr2=0x7fad4c06e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c159280 0x7fad4c06e450 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fad440098e0 tx=0x7fad440091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- 172.21.15.146:0/4144352811 shutdown_connections 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c159280 0x7fad4c06e450 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 --2- 172.21.15.146:0/4144352811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- 172.21.15.146:0/4144352811 >> 172.21.15.146:0/4144352811 conn(0x7fad4c0678d0 msgr2=0x7fad4c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- 172.21.15.146:0/4144352811 shutdown_connections 2024-08-15T17:00:15.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.945+0000 7fad4b59e700 1 -- 172.21.15.146:0/4144352811 wait complete. 2024-08-15T17:00:15.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 Processor -- start 2024-08-15T17:00:15.960 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 -- start start 2024-08-15T17:00:15.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.962 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad4c159280 0x7fad4c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0c8680 con 0x7fad4c158eb0 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0c94e0 con 0x7fad4c159280 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c145c60 con 0x7fad4c0aecc0 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55576/0 (socket says 172.21.15.146:55576) 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 -- 172.21.15.146:0/2100846280 learned_addr learned my addr 172.21.15.146:0/2100846280 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad4c159280 0x7fad4c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 -- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 msgr2=0x7fad4c145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 -- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad4c159280 msgr2=0x7fad4c157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad4c159280 0x7fad4c157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:15.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 -- 172.21.15.146:0/2100846280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad44009420 con 0x7fad4c158eb0 2024-08-15T17:00:15.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.949+0000 7fad49d9b700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00cfb0 tx=0x7fad3c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00ae10 con 0x7fad4c158eb0 2024-08-15T17:00:15.964 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c016420 con 0x7fad4c158eb0 2024-08-15T17:00:15.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c0c88d0 con 0x7fad4c158eb0 2024-08-15T17:00:15.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c0167e0 con 0x7fad4c158eb0 2024-08-15T17:00:15.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c0c8e90 con 0x7fad4c158eb0 2024-08-15T17:00:15.965 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4c0c9650 con 0x7fad4c158eb0 2024-08-15T17:00:15.966 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.953+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fad3c016bd0 con 0x7fad4c158eb0 2024-08-15T17:00:15.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.957+0000 7fad427fc700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:15.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.957+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fad3c012070 con 0x7fad4c158eb0 2024-08-15T17:00:15.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.957+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:15.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.957+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad34007130 tx=0x7fad34007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:15.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:15.961+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad3c04d870 con 0x7fad4c158eb0 2024-08-15T17:00:16.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.125+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad4c0643a0 con 0x7fad4c158eb0 2024-08-15T17:00:16.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.125+0000 7fad427fc700 1 -- 172.21.15.146:0/2100846280 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fad3c050e90 con 0x7fad4c158eb0 2024-08-15T17:00:16.138 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:16.138 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:16.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 msgr2=0x7fad2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad34007130 tx=0x7fad34007eb0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 msgr2=0x7fad4c156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00cfb0 tx=0x7fad3c007870 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 shutdown_connections 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad4c159280 0x7fad4c157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c158eb0 0x7fad4c156ae0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 --2- 172.21.15.146:0/2100846280 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0aecc0 0x7fad4c145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 >> 172.21.15.146:0/2100846280 conn(0x7fad4c0678d0 msgr2=0x7fad4c152320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 shutdown_connections 2024-08-15T17:00:16.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.129+0000 7fad4b59e700 1 -- 172.21.15.146:0/2100846280 wait complete. 2024-08-15T17:00:16.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T16:59:40.024316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T16:59:40.731449+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-15T17:00:16.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:16.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 Processor -- start 2024-08-15T17:00:16.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 -- start start 2024-08-15T17:00:16.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c159300 0x7f5f1c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f1c067510 0x7f5f1c0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c0cba90 con 0x7f5f1c1596d0 2024-08-15T17:00:16.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0b2b60 con 0x7f5f1c067510 2024-08-15T17:00:16.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.253+0000 7f5f235d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0cb160 con 0x7f5f1c159300 2024-08-15T17:00:16.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5f1c067510 0x7f5f1c0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f20b6e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5f1c1596d0 0x7f5f1c066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39484/0 (socket says 172.21.15.146:39484) 2024-08-15T17:00:16.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f20b6e700 1 -- 172.21.15.146:0/3362648654 learned_addr learned my addr 172.21.15.146:0/3362648654 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:16.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f2136f700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c159300 0x7f5f1c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f13fff700 1 -- 172.21.15.146:0/3362648654 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171200487 0 0) 0x7f5f1c0cba90 con 0x7f5f1c1596d0 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f13fff700 1 -- 172.21.15.146:0/3362648654 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f08003570 con 0x7f5f1c1596d0 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 -- 172.21.15.146:0/3362648654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c159300 msgr2=0x7f5f1c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c159300 0x7f5f1c0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 -- 172.21.15.146:0/3362648654 >> v1:172.21.15.146:6789/0 conn(0x7f5f1c1596d0 legacy=0x7f5f1c066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 -- 172.21.15.146:0/3362648654 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0cb440 con 0x7f5f1c067510 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f21b70700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c067510 0x7f5f1c0cbf20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009e60 tx=0x7f5f14009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55732a24a1dc3a4c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f13fff700 1 -- 172.21.15.146:0/3362648654 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400e070 con 0x7f5f1c067510 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f13fff700 1 -- 172.21.15.146:0/3362648654 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400bab0 con 0x7f5f1c067510 2024-08-15T17:00:16.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f13fff700 1 -- 172.21.15.146:0/3362648654 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400be70 con 0x7f5f1c067510 2024-08-15T17:00:16.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 -- 172.21.15.146:0/3362648654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c067510 msgr2=0x7f5f1c0cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c067510 0x7f5f1c0cbf20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009e60 tx=0x7f5f14009780 comp rx=0 tx=0).stop 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 -- 172.21.15.146:0/3362648654 shutdown_connections 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c067510 0x7f5f1c0cbf20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 --2- 172.21.15.146:0/3362648654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c159300 0x7f5f1c0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 -- 172.21.15.146:0/3362648654 >> 172.21.15.146:0/3362648654 conn(0x7f5f1c0682c0 msgr2=0x7f5f1c0686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 -- 172.21.15.146:0/3362648654 shutdown_connections 2024-08-15T17:00:16.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.257+0000 7f5f235d3700 1 -- 172.21.15.146:0/3362648654 wait complete. 2024-08-15T17:00:16.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 Processor -- start 2024-08-15T17:00:16.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 -- start start 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c067510 0x7f5f1c15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f1c1596d0 0x7f5f1c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.261+0000 7f5f235d3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15af20 con 0x7f5f1c159300 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f235d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15c2a0 con 0x7f5f1c1596d0 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f235d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c15aa90 con 0x7f5f1c067510 2024-08-15T17:00:16.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55598/0 (socket says 172.21.15.146:55598) 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 -- 172.21.15.146:0/2363805849 learned_addr learned my addr 172.21.15.146:0/2363805849 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f2136f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c067510 0x7f5f1c15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f21b70700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f1c1596d0 0x7f5f1c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 -- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c067510 msgr2=0x7f5f1c15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c067510 0x7f5f1c15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 -- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f1c1596d0 msgr2=0x7f5f1c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f1c1596d0 0x7f5f1c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 -- 172.21.15.146:0/2363805849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009a20 con 0x7f5f1c159300 2024-08-15T17:00:16.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f20b6e700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c590 tx=0x7f5f18007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18005b30 con 0x7f5f1c159300 2024-08-15T17:00:16.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18005cb0 con 0x7f5f1c159300 2024-08-15T17:00:16.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c14cc40 con 0x7f5f1c159300 2024-08-15T17:00:16.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18007320 con 0x7f5f1c159300 2024-08-15T17:00:16.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c15a160 con 0x7f5f1c159300 2024-08-15T17:00:16.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.265+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c0645c0 con 0x7f5f1c159300 2024-08-15T17:00:16.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.269+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5f1801b3b0 con 0x7f5f1c159300 2024-08-15T17:00:16.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.269+0000 7f5f117fa700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 0x7f5f00062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.269+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f1807bfe0 con 0x7f5f1c159300 2024-08-15T17:00:16.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.269+0000 7f5f2136f700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 0x7f5f00062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.269+0000 7f5f2136f700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 0x7f5f00062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009db0 tx=0x7f5f0c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.277+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5f1804c3b0 con 0x7f5f1c159300 2024-08-15T17:00:16.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.437+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f1c0caac0 con 0x7f5f1c159300 2024-08-15T17:00:16.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f117fa700 1 -- 172.21.15.146:0/2363805849 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f180170a0 con 0x7f5f1c159300 2024-08-15T17:00:16.452 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 msgr2=0x7f5f00062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 0x7f5f00062d10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009db0 tx=0x7f5f0c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 msgr2=0x7f5f1c14c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c590 tx=0x7f5f18007e00 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 shutdown_connections 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5f00060860 0x7f5f00062d10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5f1c1596d0 0x7f5f1c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f1c159300 0x7f5f1c14c700 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 --2- 172.21.15.146:0/2363805849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5f1c067510 0x7f5f1c15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 >> 172.21.15.146:0/2363805849 conn(0x7f5f1c0682c0 msgr2=0x7f5f1c157ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 shutdown_connections 2024-08-15T17:00:16.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.441+0000 7f5f235d3700 1 -- 172.21.15.146:0/2363805849 wait complete. 2024-08-15T17:00:16.470 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:00:16.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.569+0000 7fd1e6ea4700 1 Processor -- start 2024-08-15T17:00:16.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.569+0000 7fd1e6ea4700 1 -- start start 2024-08-15T17:00:16.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1e0152bd0 0x7fd1e0153010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e00ace30 con 0x7fd1e00b1830 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1e00b1c80 con 0x7fd1e0152bd0 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00ae6b0 con 0x7fd1e006ead0 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1e0152bd0 0x7fd1e0153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1e0152bd0 0x7fd1e0153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:60024/0 (socket says 172.21.15.146:60024) 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 -- 172.21.15.146:0/957810606 learned_addr learned my addr 172.21.15.146:0/957810606 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e56a1700 1 --1- 172.21.15.146:0/957810606 >> v1:172.21.15.161:6789/0 conn(0x7fd1e00b1830 0x7fd1e0155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:39346/0 (socket says 172.21.15.146:39346) 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e5ea2700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 -- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 msgr2=0x7fd1e006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e006eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 -- 172.21.15.146:0/957810606 >> v1:172.21.15.161:6789/0 conn(0x7fd1e00b1830 legacy=0x7fd1e0155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 -- 172.21.15.146:0/957810606 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0156dd0 con 0x7fd1e0152bd0 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e66a3700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0153010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00099b0 tx=0x7fd1d0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b1af1abff76335b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e4ea0700 1 -- 172.21.15.146:0/957810606 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0009ea0 con 0x7fd1e0152bd0 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e4ea0700 1 -- 172.21.15.146:0/957810606 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d000b860 con 0x7fd1e0152bd0 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e4ea0700 1 -- 172.21.15.146:0/957810606 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000bc00 con 0x7fd1e0152bd0 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 msgr2=0x7fd1e0153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0153010 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00099b0 tx=0x7fd1d0009250 comp rx=0 tx=0).stop 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/957810606 shutdown_connections 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0153010 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/957810606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e006eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/957810606 >> 172.21.15.146:0/957810606 conn(0x7fd1e00678a0 msgr2=0x7fd1e0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/957810606 shutdown_connections 2024-08-15T17:00:16.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.573+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/957810606 wait complete. 2024-08-15T17:00:16.591 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 Processor -- start 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- start start 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e0151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0146d50 con 0x7fd1e00b1830 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0144620 con 0x7fd1e0152bd0 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0151960 con 0x7fd1e006ead0 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55608/0 (socket says 172.21.15.146:55608) 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 -- 172.21.15.146:0/1702621045 learned_addr learned my addr 172.21.15.146:0/1702621045 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e5ea2700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e0151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e66a3700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 -- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 msgr2=0x7fd1e0151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e0151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 -- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 msgr2=0x7fd1e0145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 -- 172.21.15.146:0/1702621045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00094f0 con 0x7fd1e00b1830 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e56a1700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00cd30 tx=0x7fd1dc001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00ce90 con 0x7fd1e00b1830 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e0151c40 con 0x7fd1e00b1830 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc005df0 con 0x7fd1e00b1830 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e0152200 con 0x7fd1e00b1830 2024-08-15T17:00:16.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.577+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc00aae0 con 0x7fd1e00b1830 2024-08-15T17:00:16.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.581+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c00052b0 con 0x7fd1e00b1830 2024-08-15T17:00:16.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.581+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd1dc0070f0 con 0x7fd1e00b1830 2024-08-15T17:00:16.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.581+0000 7fd1d5ffb700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 0x7fd1cc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.585+0000 7fd1e5ea2700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 0x7fd1cc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.585+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd1dc07cbd0 con 0x7fd1e00b1830 2024-08-15T17:00:16.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.585+0000 7fd1e5ea2700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 0x7fd1cc062d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009d10 tx=0x7fd1d80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.593+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd1dc04d000 con 0x7fd1e00b1830 2024-08-15T17:00:16.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.753+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1c0005ae0 con 0x7fd1e00b1830 2024-08-15T17:00:16.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1d5ffb700 1 -- 172.21.15.146:0/1702621045 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1dc050620 con 0x7fd1e00b1830 2024-08-15T17:00:16.767 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 msgr2=0x7fd1cc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 0x7fd1cc062d10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009d10 tx=0x7fd1d80091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 msgr2=0x7fd1e01440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00cd30 tx=0x7fd1dc001af0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 shutdown_connections 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1cc060860 0x7fd1cc062d10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1e0152bd0 0x7fd1e0145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1e00b1830 0x7fd1e01440e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 --2- 172.21.15.146:0/1702621045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1e006ead0 0x7fd1e0151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 >> 172.21.15.146:0/1702621045 conn(0x7fd1e00678a0 msgr2=0x7fd1e0069610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 shutdown_connections 2024-08-15T17:00:16.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.757+0000 7fd1e6ea4700 1 -- 172.21.15.146:0/1702621045 wait complete. 2024-08-15T17:00:16.782 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:00:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 Processor -- start 2024-08-15T17:00:16.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 -- start start 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f959c156e90 0x7f959c0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c06da00 0x7f959c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0cc580 con 0x7f959c157260 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959c0cc860 con 0x7f959c156e90 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f95a0ed2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0ccb40 con 0x7f959c06da00 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f9599d9b700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f959c157260 0x7f959c06d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39494/0 (socket says 172.21.15.146:39494) 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f9599d9b700 1 -- 172.21.15.146:0/4076858550 learned_addr learned my addr 172.21.15.146:0/4076858550 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.885+0000 7f959a59c700 1 --2- 172.21.15.146:0/4076858550 >> v2:172.21.15.146:3300/0 conn(0x7f959c156e90 0x7f959c0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959ad9d700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c06da00 0x7f959c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959959a700 1 -- 172.21.15.146:0/4076858550 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229884906 0 0) 0x7f959c0cc580 con 0x7f959c157260 2024-08-15T17:00:16.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959959a700 1 -- 172.21.15.146:0/4076858550 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9584003570 con 0x7f959c157260 2024-08-15T17:00:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959a59c700 1 -- 172.21.15.146:0/4076858550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c06da00 msgr2=0x7f959c0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959a59c700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c06da00 0x7f959c0b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959a59c700 1 -- 172.21.15.146:0/4076858550 >> v1:172.21.15.146:6789/0 conn(0x7f959c157260 legacy=0x7f959c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959a59c700 1 -- 172.21.15.146:0/4076858550 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0cce20 con 0x7f959c156e90 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959a59c700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c156e90 0x7f959c0cbcc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f95900098e0 tx=0x7f95900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b5dfb2580d6635 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959959a700 1 -- 172.21.15.146:0/4076858550 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590011630 con 0x7f959c156e90 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959959a700 1 -- 172.21.15.146:0/4076858550 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590011790 con 0x7f959c156e90 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f959959a700 1 -- 172.21.15.146:0/4076858550 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590011b10 con 0x7f959c156e90 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4076858550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c156e90 msgr2=0x7f959c0cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c156e90 0x7f959c0cbcc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f95900098e0 tx=0x7f95900091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4076858550 shutdown_connections 2024-08-15T17:00:16.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c06da00 0x7f959c0b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4076858550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c156e90 0x7f959c0cbcc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4076858550 >> 172.21.15.146:0/4076858550 conn(0x7f959c0ceb30 msgr2=0x7f959c06cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4076858550 shutdown_connections 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.889+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4076858550 wait complete. 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 Processor -- start 2024-08-15T17:00:16.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- start start 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 0x7f959c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c0687a0 0x7f959c151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c14eb50 con 0x7f959c06da00 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c14dcc0 con 0x7f959c157260 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c14df40 con 0x7f959c0687a0 2024-08-15T17:00:16.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55614/0 (socket says 172.21.15.146:55614) 2024-08-15T17:00:16.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 -- 172.21.15.146:0/4186540880 learned_addr learned my addr 172.21.15.146:0/4186540880 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:16.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f9599d9b700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 0x7f959c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959ad9d700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c0687a0 0x7f959c151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 -- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c0687a0 msgr2=0x7f959c151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c0687a0 0x7f959c151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 -- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 msgr2=0x7f959c150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 0x7f959c150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 -- 172.21.15.146:0/4186540880 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9590009420 con 0x7f959c06da00 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f9599d9b700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 0x7f959c150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f959a59c700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f959c06eba0 tx=0x7f9590017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590002f90 con 0x7f959c06da00 2024-08-15T17:00:16.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590011900 con 0x7f959c06da00 2024-08-15T17:00:16.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c14e220 con 0x7f959c06da00 2024-08-15T17:00:16.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959001a660 con 0x7f959c06da00 2024-08-15T17:00:16.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.893+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c14e7e0 con 0x7f959c06da00 2024-08-15T17:00:16.908 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.897+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c069fd0 con 0x7f959c06da00 2024-08-15T17:00:16.909 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.897+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f959001aa90 con 0x7f959c06da00 2024-08-15T17:00:16.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.897+0000 7f958a7fc700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 0x7f957c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:16.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.897+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f959007e6d0 con 0x7f959c06da00 2024-08-15T17:00:16.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.897+0000 7f9599d9b700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 0x7f957c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:16.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.901+0000 7f9599d9b700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 0x7f957c062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f959c156860 tx=0x7f958c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:16.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:16.905+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f959c069fd0 con 0x7f959c06da00 2024-08-15T17:00:17.049 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:00:17.051 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:00:17.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.069+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f959c069fd0 con 0x7f959c06da00 2024-08-15T17:00:17.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.069+0000 7f958a7fc700 1 -- 172.21.15.146:0/4186540880 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f959c069fd0 con 0x7f959c06da00 2024-08-15T17:00:17.082 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 msgr2=0x7f957c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 0x7f957c062d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f959c156860 tx=0x7f958c009040 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 msgr2=0x7f959c14bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f959c06eba0 tx=0x7f9590017000 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 shutdown_connections 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f957c060860 0x7f957c062d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f959c0687a0 0x7f959c151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f959c157260 0x7f959c150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 --2- 172.21.15.146:0/4186540880 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f959c06da00 0x7f959c14bbb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 >> 172.21.15.146:0/4186540880 conn(0x7f959c0ceb30 msgr2=0x7f959c15d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 shutdown_connections 2024-08-15T17:00:17.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.073+0000 7f95a0ed2700 1 -- 172.21.15.146:0/4186540880 wait complete. 2024-08-15T17:00:17.096 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:17.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 Processor -- start 2024-08-15T17:00:17.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 -- start start 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6f0406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f04155720 0x7f6f040b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f040aba70 con 0x7f6f040ab540 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f040abd50 con 0x7f6f04155720 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f09359700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040ac030 con 0x7f6f0406e9d0 2024-08-15T17:00:17.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6f040ab540 0x7f6f04155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39504/0 (socket says 172.21.15.146:39504) 2024-08-15T17:00:17.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 -- 172.21.15.146:0/2049721577 learned_addr learned my addr 172.21.15.146:0/2049721577 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:17.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f08b58700 1 --2- 172.21.15.146:0/2049721577 >> v2:172.21.15.161:3300/0 conn(0x7f6f04155720 0x7f6f040b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569276494 0 0) 0x7f6f040aba70 con 0x7f6f040ab540 2024-08-15T17:00:17.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f03fff700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6f0406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee4003570 con 0x7f6f040ab540 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899027819 0 0) 0x7f6ee4003570 con 0x7f6f040ab540 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 msgr2=0x7f6f0406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6f0406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 >> v2:172.21.15.161:3300/0 conn(0x7f6f04155720 msgr2=0x7f6f040b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 --2- 172.21.15.146:0/2049721577 >> v2:172.21.15.161:3300/0 conn(0x7f6f04155720 0x7f6f040b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040ac310 con 0x7f6f040ab540 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f6eec001470 con 0x7f6f040ab540 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 >> v1:172.21.15.146:6789/0 conn(0x7f6f040ab540 legacy=0x7f6f04155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6ee4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6ee4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee4004d50 0x7f6ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040abd50 con 0x7f6ee4003fe0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040ac030 con 0x7f6ee4004d50 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4005b10 con 0x7f6f0406e9d0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6ee4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f08b58700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee4004d50 0x7f6ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f03fff700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6ee4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 -- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 msgr2=0x7f6ee4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6ee4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 -- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee4004d50 msgr2=0x7f6ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6ee4004d50 0x7f6ee40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 -- 172.21.15.146:0/2049721577 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4003570 con 0x7f6ee4003fe0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f03fff700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0406e9d0 0x7f6ee4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f037fe700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6ee4004810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c390 tx=0x7f6f040ac5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da386f6df558f020 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.241+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec00cc60 con 0x7f6ee4003fe0 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eec00cdc0 con 0x7f6ee4003fe0 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f02ffd700 1 -- 172.21.15.146:0/2049721577 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eec005b70 con 0x7f6ee4003fe0 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- 172.21.15.146:0/2049721577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 msgr2=0x7f6ee4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 --2- 172.21.15.146:0/2049721577 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6ee4004810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c390 tx=0x7f6f040ac5f0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f08b58700 1 -- 172.21.15.146:0/2049721577 reap_dead start 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- 172.21.15.146:0/2049721577 shutdown_connections 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- 172.21.15.146:0/2049721577 >> 172.21.15.146:0/2049721577 conn(0x7f6f040678d0 msgr2=0x7f6f04067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- 172.21.15.146:0/2049721577 shutdown_connections 2024-08-15T17:00:17.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- 172.21.15.146:0/2049721577 wait complete. 2024-08-15T17:00:17.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 Processor -- start 2024-08-15T17:00:17.260 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.245+0000 7f6f09359700 1 -- start start 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f0415c670 0x7f6f040a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 0x7f6f040a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f040a8a70 con 0x7f6ee4003fe0 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415d080 con 0x7f6f0415c670 2024-08-15T17:00:17.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0415d360 con 0x7f6f0415cae0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55634/0 (socket says 172.21.15.146:55634) 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 -- 172.21.15.146:0/44500375 learned_addr learned my addr 172.21.15.146:0/44500375 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f037fe700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f0415c670 0x7f6f040a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f08b58700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 0x7f6f040a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 -- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 msgr2=0x7f6f040a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 0x7f6f040a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 -- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f0415c670 msgr2=0x7f6f040a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f0415c670 0x7f6f040a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 -- 172.21.15.146:0/44500375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0415d640 con 0x7f6ee4003fe0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f08b58700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 0x7f6f040a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f03fff700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400e8d0 tx=0x7f6ef4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400ee70 con 0x7f6ee4003fe0 2024-08-15T17:00:17.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400c5e0 con 0x7f6ee4003fe0 2024-08-15T17:00:17.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f040a8700 con 0x7f6ee4003fe0 2024-08-15T17:00:17.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400c9a0 con 0x7f6ee4003fe0 2024-08-15T17:00:17.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.249+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0415ec50 con 0x7f6ee4003fe0 2024-08-15T17:00:17.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f040a8560 con 0x7f6ee4003fe0 2024-08-15T17:00:17.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ef4013410 con 0x7f6ee4003fe0 2024-08-15T17:00:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6efbfff700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 0x7f6ee8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ef407cac0 con 0x7f6ee4003fe0 2024-08-15T17:00:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6f037fe700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 0x7f6ee8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.253+0000 7f6f037fe700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 0x7f6ee8062d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f0415daf0 tx=0x7f6eec015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.261+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6f040a8560 con 0x7f6ee4003fe0 2024-08-15T17:00:17.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.457+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f040a8560 con 0x7f6ee4003fe0 2024-08-15T17:00:17.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6efbfff700 1 -- 172.21.15.146:0/44500375 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f6f040a8560 con 0x7f6ee4003fe0 2024-08-15T17:00:17.472 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:17.472 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:00:17.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 msgr2=0x7f6ee8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 0x7f6ee8062d10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f0415daf0 tx=0x7f6eec015000 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 msgr2=0x7f6f04150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400e8d0 tx=0x7f6ef4008040 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 shutdown_connections 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ee8060860 0x7f6ee8062d10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f0415cae0 0x7f6f040a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f0415c670 0x7f6f040a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 --2- 172.21.15.146:0/44500375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6ee4003fe0 0x7f6f04150280 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 >> 172.21.15.146:0/44500375 conn(0x7f6f040678d0 msgr2=0x7f6f0406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 shutdown_connections 2024-08-15T17:00:17.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.461+0000 7f6f09359700 1 -- 172.21.15.146:0/44500375 wait complete. 2024-08-15T17:00:17.476 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T17:00:17.486 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:17.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 Processor -- start 2024-08-15T17:00:17.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- start start 2024-08-15T17:00:17.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f537015d540 0x7f53700af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f537006e830 0x7f53700a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53700b0270 con 0x7f53700a9260 2024-08-15T17:00:17.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700af660 con 0x7f537015d540 2024-08-15T17:00:17.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700af940 con 0x7f537006e830 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f537015d540 0x7f53700af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f537015d540 0x7f53700af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55638/0 (socket says 172.21.15.146:55638) 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 -- 172.21.15.146:0/2218730653 learned_addr learned my addr 172.21.15.146:0/2218730653 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5375a83700 1 --2- 172.21.15.146:0/2218730653 >> v2:172.21.15.161:3300/0 conn(0x7f537006e830 0x7f53700a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 -- 172.21.15.146:0/2218730653 >> v2:172.21.15.161:3300/0 conn(0x7f537006e830 msgr2=0x7f53700a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 --2- 172.21.15.146:0/2218730653 >> v2:172.21.15.161:3300/0 conn(0x7f537006e830 0x7f53700a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 -- 172.21.15.146:0/2218730653 >> v1:172.21.15.161:6789/0 conn(0x7f53700a9260 legacy=0x7f53700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 -- 172.21.15.146:0/2218730653 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53700afc20 con 0x7f537015d540 2024-08-15T17:00:17.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5376284700 1 --2- 172.21.15.146:0/2218730653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537015d540 0x7f53700af030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f537006c970 tx=0x7f536c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab2dcad3ea0efb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5375282700 1 -- 172.21.15.146:0/2218730653 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c00e070 con 0x7f537015d540 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5375282700 1 -- 172.21.15.146:0/2218730653 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c00ba20 con 0x7f537015d540 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f5375282700 1 -- 172.21.15.146:0/2218730653 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0092e0 con 0x7f537015d540 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- 172.21.15.146:0/2218730653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537015d540 msgr2=0x7f53700af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 --2- 172.21.15.146:0/2218730653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537015d540 0x7f53700af030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f537006c970 tx=0x7f536c009840 comp rx=0 tx=0).stop 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- 172.21.15.146:0/2218730653 shutdown_connections 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 --2- 172.21.15.146:0/2218730653 >> v2:172.21.15.161:3300/0 conn(0x7f537006e830 0x7f53700a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 --2- 172.21.15.146:0/2218730653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537015d540 0x7f53700af030 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- 172.21.15.146:0/2218730653 >> 172.21.15.146:0/2218730653 conn(0x7f5370066e00 msgr2=0x7f5370067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- 172.21.15.146:0/2218730653 shutdown_connections 2024-08-15T17:00:17.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.577+0000 7f53784e8700 1 -- 172.21.15.146:0/2218730653 wait complete. 2024-08-15T17:00:17.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 Processor -- start 2024-08-15T17:00:17.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 -- start start 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53700a9260 0x7f5370098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f537015d540 0x7f5370159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53701583d0 con 0x7f537006e830 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5370096e40 con 0x7f537015d540 2024-08-15T17:00:17.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f53784e8700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5370097120 con 0x7f53700a9260 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f5376284700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f5376284700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55644/0 (socket says 172.21.15.146:55644) 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f5376284700 1 -- 172.21.15.146:0/838441050 learned_addr learned my addr 172.21.15.146:0/838441050 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f5375a83700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53700a9260 0x7f5370098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.581+0000 7f5376a85700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f537015d540 0x7f5370159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 -- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53700a9260 msgr2=0x7f5370098980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53700a9260 0x7f5370098980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 -- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f537015d540 msgr2=0x7f5370159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f537015d540 0x7f5370159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 -- 172.21.15.146:0/838441050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c009ae0 con 0x7f537006e830 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f5376284700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f537006f870 tx=0x7f536c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0078f0 con 0x7f537006e830 2024-08-15T17:00:17.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c007a70 con 0x7f537006e830 2024-08-15T17:00:17.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5370097400 con 0x7f537006e830 2024-08-15T17:00:17.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c007e30 con 0x7f537006e830 2024-08-15T17:00:17.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5370097a40 con 0x7f537006e830 2024-08-15T17:00:17.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53700645c0 con 0x7f537006e830 2024-08-15T17:00:17.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.585+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f536c003040 con 0x7f537006e830 2024-08-15T17:00:17.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.589+0000 7f53667fc700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 0x7f5354062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.589+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f536c07e650 con 0x7f537006e830 2024-08-15T17:00:17.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.589+0000 7f5375a83700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 0x7f5354062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.589+0000 7f5375a83700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 0x7f5354062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f537006a160 tx=0x7f53600091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.593+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f536c04eaa0 con 0x7f537006e830 2024-08-15T17:00:17.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.761+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f537006ad90 con 0x7f537006e830 2024-08-15T17:00:17.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.761+0000 7f53667fc700 1 -- 172.21.15.146:0/838441050 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f536c0520c0 con 0x7f537006e830 2024-08-15T17:00:17.775 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:17.775 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 msgr2=0x7f5354062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 0x7f5354062d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f537006a160 tx=0x7f53600091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 msgr2=0x7f5370157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f537006f870 tx=0x7f536c018000 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 shutdown_connections 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5354060860 0x7f5354062d10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f537015d540 0x7f5370159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53700a9260 0x7f5370098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 --2- 172.21.15.146:0/838441050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f537006e830 0x7f5370157e90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 >> 172.21.15.146:0/838441050 conn(0x7f5370066e00 msgr2=0x7f537015ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 shutdown_connections 2024-08-15T17:00:17.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.765+0000 7f53784e8700 1 -- 172.21.15.146:0/838441050 wait complete. 2024-08-15T17:00:17.788 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:00:17.789 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:17.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 Processor -- start 2024-08-15T17:00:17.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 -- start start 2024-08-15T17:00:17.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f97840b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97840ae8c0 con 0x7f978413fa40 2024-08-15T17:00:17.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97840b1820 con 0x7f978406e720 2024-08-15T17:00:17.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978b061700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840ad710 con 0x7f978413f670 2024-08-15T17:00:17.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978985e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f978413fa40 0x7f97840ade00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:39520/0 (socket says 172.21.15.146:39520) 2024-08-15T17:00:17.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978985e700 1 -- 172.21.15.146:0/3689070043 learned_addr learned my addr 172.21.15.146:0/3689070043 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:17.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f97840b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164979191 0 0) 0x7f97840ae8c0 con 0x7f978413fa40 2024-08-15T17:00:17.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9770003570 con 0x7f978413fa40 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283385919 0 0) 0x7f9770003570 con 0x7f978413fa40 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 msgr2=0x7f97840b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f97840b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 >> v1:172.21.15.161:6789/0 conn(0x7f978406e720 legacy=0x7f978406eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840b0ef0 con 0x7f978413fa40 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f9780001470 con 0x7f978413fa40 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 >> v1:172.21.15.146:6789/0 conn(0x7f978413fa40 legacy=0x7f97840ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9770003a90 0x7f9770003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f9770004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770004d50 0x7f97700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840ad710 con 0x7f9770003a90 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977000bb10 con 0x7f9770004d50 2024-08-15T17:00:17.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977000bcd0 con 0x7f978413f670 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9770003a90 0x7f9770003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978985e700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f9770004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a860700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770004d50 0x7f97700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 -- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 msgr2=0x7f9770004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978413f670 0x7f9770004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 -- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770004d50 msgr2=0x7f97700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770004d50 0x7f97700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 -- 172.21.15.146:0/3689070043 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770003570 con 0x7f9770003a90 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.889+0000 7f978a05f700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9770003a90 0x7f9770003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9774009c00 tx=0x7f97740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7162cd1dd5876139 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977400e710 con 0x7f9770003a90 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977400e870 con 0x7f9770003a90 2024-08-15T17:00:17.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978905d700 1 -- 172.21.15.146:0/3689070043 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977400ec30 con 0x7f9770003a90 2024-08-15T17:00:17.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 -- 172.21.15.146:0/3689070043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9770003a90 msgr2=0x7f9770003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 --2- 172.21.15.146:0/3689070043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9770003a90 0x7f9770003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9774009c00 tx=0x7f97740091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978a860700 1 -- 172.21.15.146:0/3689070043 reap_dead start 2024-08-15T17:00:17.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 -- 172.21.15.146:0/3689070043 shutdown_connections 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 -- 172.21.15.146:0/3689070043 >> 172.21.15.146:0/3689070043 conn(0x7f97840ca2b0 msgr2=0x7f978406e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 -- 172.21.15.146:0/3689070043 shutdown_connections 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.893+0000 7f978b061700 1 -- 172.21.15.146:0/3689070043 wait complete. 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 Processor -- start 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 -- start start 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 0x7f978413d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a69d0 con 0x7f97840a6e60 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a5dc0 con 0x7f9770003a90 2024-08-15T17:00:17.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978b061700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840a60a0 con 0x7f978406af20 2024-08-15T17:00:17.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978985e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.913 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978985e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:35234/0 (socket says 172.21.15.146:35234) 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a05f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 0x7f978413d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:55668/0 (socket says 172.21.15.146:55668) 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978985e700 1 -- 172.21.15.146:0/850870617 learned_addr learned my addr 172.21.15.146:0/850870617 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 -- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 msgr2=0x7f97840a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 -- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 msgr2=0x7f978413d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 0x7f978413d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 -- 172.21.15.146:0/850870617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840a6380 con 0x7f97840a6e60 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a05f700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 0x7f978413d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978985e700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.897+0000 7f978a860700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9784070c70 tx=0x7f977c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.914 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0032e0 con 0x7f97840a6e60 2024-08-15T17:00:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c003480 con 0x7f97840a6e60 2024-08-15T17:00:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97840a6660 con 0x7f97840a6e60 2024-08-15T17:00:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c006df0 con 0x7f97840a6e60 2024-08-15T17:00:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97841567a0 con 0x7f97840a6e60 2024-08-15T17:00:17.915 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97840643a0 con 0x7f97840a6e60 2024-08-15T17:00:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f977c001850 con 0x7f97840a6e60 2024-08-15T17:00:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 0x7f9768062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f977c07dfa0 con 0x7f97840a6e60 2024-08-15T17:00:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.901+0000 7f978a05f700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 0x7f9768062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:17.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.905+0000 7f978a05f700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 0x7f9768062d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97740091e0 tx=0x7f977401a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:17.923 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:17.909+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f977c04e3f0 con 0x7f97840a6e60 2024-08-15T17:00:18.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.109+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97840cb240 con 0x7f97840a6e60 2024-08-15T17:00:18.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.109+0000 7f977a7fc700 1 -- 172.21.15.146:0/850870617 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f977c01b090 con 0x7f97840a6e60 2024-08-15T17:00:18.126 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:18.126 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:00:18.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 msgr2=0x7f9768062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 0x7f9768062d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97740091e0 tx=0x7f977401a000 comp rx=0 tx=0).stop 2024-08-15T17:00:18.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 msgr2=0x7f9784156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9784070c70 tx=0x7f977c009040 comp rx=0 tx=0).stop 2024-08-15T17:00:18.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 shutdown_connections 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9768060890 0x7f9768062d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f97840a6e60 0x7f9784156260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f978406af20 0x7f97840a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 --2- 172.21.15.146:0/850870617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9770003a90 0x7f978413d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 >> 172.21.15.146:0/850870617 conn(0x7f97840ca2b0 msgr2=0x7f9784156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 shutdown_connections 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.113+0000 7f978b061700 1 -- 172.21.15.146:0/850870617 wait complete. 2024-08-15T17:00:18.129 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T17:00:18.136 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:18.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.225+0000 7f9e8759e700 1 Processor -- start 2024-08-15T17:00:18.239 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.225+0000 7f9e8759e700 1 -- start start 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e880cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 0x7f9e880d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e880cf190 con 0x7f9e8815cea0 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880d09a0 con 0x7f9e880d0010 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880ce5e0 con 0x7f9e88159040 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e85d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 0x7f9e880d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e880cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e85d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 0x7f9e880d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:50294/0 (socket says 172.21.15.146:50294) 2024-08-15T17:00:18.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e85d9b700 1 -- 172.21.15.146:0/3771804865 learned_addr learned my addr 172.21.15.146:0/3771804865 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:18.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e86d9d700 1 --1- 172.21.15.146:0/3771804865 >> v1:172.21.15.161:6789/0 conn(0x7f9e8815cea0 0x7f9e8815d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:49146/0 (socket says 172.21.15.146:49146) 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 -- 172.21.15.146:0/3771804865 >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 msgr2=0x7f9e880d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 --2- 172.21.15.146:0/3771804865 >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 0x7f9e880d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 -- 172.21.15.146:0/3771804865 >> v1:172.21.15.161:6789/0 conn(0x7f9e8815cea0 legacy=0x7f9e8815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 -- 172.21.15.146:0/3771804865 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880ce860 con 0x7f9e88159040 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8659c700 1 --2- 172.21.15.146:0/3771804865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e880cf620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009e60 tx=0x7f9e78009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283477d3787c8a2d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8559a700 1 -- 172.21.15.146:0/3771804865 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800e070 con 0x7f9e88159040 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8559a700 1 -- 172.21.15.146:0/3771804865 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7800bb60 con 0x7f9e88159040 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8559a700 1 -- 172.21.15.146:0/3771804865 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78012430 con 0x7f9e88159040 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 -- 172.21.15.146:0/3771804865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 msgr2=0x7f9e880cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.229+0000 7f9e8759e700 1 --2- 172.21.15.146:0/3771804865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e880cf620 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009e60 tx=0x7f9e78009780 comp rx=0 tx=0).stop 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- 172.21.15.146:0/3771804865 shutdown_connections 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 --2- 172.21.15.146:0/3771804865 >> v2:172.21.15.161:3300/0 conn(0x7f9e880d0010 0x7f9e880d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 --2- 172.21.15.146:0/3771804865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e880cf620 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- 172.21.15.146:0/3771804865 >> 172.21.15.146:0/3771804865 conn(0x7f9e88068250 msgr2=0x7f9e88068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- 172.21.15.146:0/3771804865 shutdown_connections 2024-08-15T17:00:18.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- 172.21.15.146:0/3771804865 wait complete. 2024-08-15T17:00:18.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 Processor -- start 2024-08-15T17:00:18.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- start start 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e880d0010 0x7f9e880b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e88151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b1df0 con 0x7f9e8815cea0 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b11e0 con 0x7f9e880d0010 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8759e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b14c0 con 0x7f9e88159040 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e86d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e86d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37724/0 (socket says 172.21.15.146:37724) 2024-08-15T17:00:18.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e86d9d700 1 -- 172.21.15.146:0/2020865814 learned_addr learned my addr 172.21.15.146:0/2020865814 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e8659c700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e880d0010 0x7f9e880b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.233+0000 7f9e85d9b700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e88151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 -- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 msgr2=0x7f9e88151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e88151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 -- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e880d0010 msgr2=0x7f9e880b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e880d0010 0x7f9e880b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 -- 172.21.15.146:0/2020865814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78009a20 con 0x7f9e8815cea0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e86d9d700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c4e0 tx=0x7f9e80007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000ce90 con 0x7f9e8815cea0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e80005a70 con 0x7f9e8815cea0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80005e10 con 0x7f9e8815cea0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880b17a0 con 0x7f9e8815cea0 2024-08-15T17:00:18.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e880b27c0 con 0x7f9e8815cea0 2024-08-15T17:00:18.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e880643a0 con 0x7f9e8815cea0 2024-08-15T17:00:18.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9e800072e0 con 0x7f9e8815cea0 2024-08-15T17:00:18.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 0x7f9e68062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e8007be70 con 0x7f9e8815cea0 2024-08-15T17:00:18.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.237+0000 7f9e8659c700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 0x7f9e68062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.241+0000 7f9e8659c700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 0x7f9e68062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e780097c0 tx=0x7f9e78019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.249+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f9e8004c2c0 con 0x7f9e8815cea0 2024-08-15T17:00:18.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.449+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e880b3480 con 0x7f9e8815cea0 2024-08-15T17:00:18.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e7e7fc700 1 -- 172.21.15.146:0/2020865814 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f9e8004f8e0 con 0x7f9e8815cea0 2024-08-15T17:00:18.464 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:18.464 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/157429633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":157429633},{"type":"v1","addr":"172.21.15.161:6837","nonce":157429633}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1593508905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1593508905},{"type":"v1","addr":"172.21.15.146:6833","nonce":1593508905}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/1081356991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":1081356991},{"type":"v1","addr":"172.21.15.146:6835","nonce":1081356991}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T16:59:41.727299+0000","modified":"2024-08-15T16:59:44.405527+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/395806260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":395806260},{"type":"v1","addr":"172.21.15.161:6835","nonce":395806260}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-15T17:00:18.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 msgr2=0x7f9e68062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 0x7f9e68062d10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e780097c0 tx=0x7f9e78019000 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 msgr2=0x7f9e880ae230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000c4e0 tx=0x7f9e80007f50 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 shutdown_connections 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9e68060860 0x7f9e68062d10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e8815cea0 0x7f9e880ae230 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e88159040 0x7f9e88151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 --2- 172.21.15.146:0/2020865814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e880d0010 0x7f9e880b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 >> 172.21.15.146:0/2020865814 conn(0x7f9e88068250 msgr2=0x7f9e88156ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 shutdown_connections 2024-08-15T17:00:18.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.453+0000 7f9e8759e700 1 -- 172.21.15.146:0/2020865814 wait complete. 2024-08-15T17:00:18.468 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 6 2024-08-15T17:00:18.479 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:18.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 Processor -- start 2024-08-15T17:00:18.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 -- start start 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4af80af200 0x7f4af81685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af80ae660 con 0x7f4af815d960 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af80c99c0 con 0x7f4af815d590 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4afd799700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af80c9f80 con 0x7f4af80af200 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4aee7fc700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4af815d960 0x7f4af80aeaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45258/0 (socket says 172.21.15.146:45258) 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4aee7fc700 1 -- 172.21.15.146:0/2602624737 learned_addr learned my addr 172.21.15.146:0/2602624737 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af77fe700 1 --2- 172.21.15.146:0/2602624737 >> v2:172.21.15.161:3300/0 conn(0x7f4af80af200 0x7f4af81685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829652405 0 0) 0x7f4af80ae660 con 0x7f4af815d960 2024-08-15T17:00:18.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae4003570 con 0x7f4af815d960 2024-08-15T17:00:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216239129 0 0) 0x7f4ae4003570 con 0x7f4af815d960 2024-08-15T17:00:18.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 >> v2:172.21.15.161:3300/0 conn(0x7f4af80af200 msgr2=0x7f4af81685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 --2- 172.21.15.146:0/2602624737 >> v2:172.21.15.161:3300/0 conn(0x7f4af80af200 0x7f4af81685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 >> v1:172.21.15.161:6789/0 conn(0x7f4af815d590 legacy=0x7f4af80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80c9ca0 con 0x7f4af815d960 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f4ae0001470 con 0x7f4af815d960 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 >> v1:172.21.15.146:6789/0 conn(0x7f4af815d960 legacy=0x7f4af80aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 0x7f4ae4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4004810 0x7f4ae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae40051c0 0x7f4ae400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80c9f80 con 0x7f4ae4003a60 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400c0d0 con 0x7f4ae40051c0 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400c290 con 0x7f4ae4004810 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 0x7f4ae4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4aee7fc700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4004810 0x7f4ae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af77fe700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae40051c0 0x7f4ae400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 -- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4004810 msgr2=0x7f4ae4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4004810 0x7f4ae4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 -- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae40051c0 msgr2=0x7f4ae400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ae40051c0 0x7f4ae400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af77fe700 1 -- 172.21.15.146:0/2602624737 reap_dead start 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 -- 172.21.15.146:0/2602624737 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4003570 con 0x7f4ae4003a60 2024-08-15T17:00:18.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 0x7f4ae4003e70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80018b0 tx=0x7f4ae8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73b89b24eafa2c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.573+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8012420 con 0x7f4ae4003a60 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae80125a0 con 0x7f4ae4003a60 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4af67fc700 1 -- 172.21.15.146:0/2602624737 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8012960 con 0x7f4ae4003a60 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 msgr2=0x7f4ae4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 0x7f4ae4003e70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80018b0 tx=0x7f4ae8007d30 comp rx=0 tx=0).stop 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- 172.21.15.146:0/2602624737 shutdown_connections 2024-08-15T17:00:18.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 --2- 172.21.15.146:0/2602624737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ae4003a60 0x7f4ae4003e70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- 172.21.15.146:0/2602624737 >> 172.21.15.146:0/2602624737 conn(0x7f4af80a9db0 msgr2=0x7f4af80ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- 172.21.15.146:0/2602624737 shutdown_connections 2024-08-15T17:00:18.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- 172.21.15.146:0/2602624737 wait complete. 2024-08-15T17:00:18.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 Processor -- start 2024-08-15T17:00:18.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.577+0000 7f4afd799700 1 -- start start 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4003a60 0x7f4af8171bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4af809a900 0x7f4af8097a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8097f90 con 0x7f4af815c1c0 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af809a470 con 0x7f4af809a900 2024-08-15T17:00:18.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80981a0 con 0x7f4ae4003a60 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37748/0 (socket says 172.21.15.146:37748) 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 -- 172.21.15.146:0/1330513836 learned_addr learned my addr 172.21.15.146:0/1330513836 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4aee7fc700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4af809a900 0x7f4af8097a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 -- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4003a60 msgr2=0x7f4af8171bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4003a60 0x7f4af8171bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 -- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4af809a900 msgr2=0x7f4af8097a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4af809a900 0x7f4af8097a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 -- 172.21.15.146:0/1330513836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8099b40 con 0x7f4af815c1c0 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4af77fe700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4af00094d0 tx=0x7f4af000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000e040 con 0x7f4af815c1c0 2024-08-15T17:00:18.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000f040 con 0x7f4af815c1c0 2024-08-15T17:00:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af8099e20 con 0x7f4af815c1c0 2024-08-15T17:00:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000fe50 con 0x7f4af815c1c0 2024-08-15T17:00:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.581+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af815c630 con 0x7f4af815c1c0 2024-08-15T17:00:18.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af80b00b0 con 0x7f4af815c1c0 2024-08-15T17:00:18.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4af0015640 con 0x7f4af815c1c0 2024-08-15T17:00:18.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4aef7fe700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 0x7f4ad8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4af007d3e0 con 0x7f4af815c1c0 2024-08-15T17:00:18.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 0x7f4ad8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.585+0000 7f4af6ffd700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 0x7f4ad8062d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8001690 tx=0x7f4ae8015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.593+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4af80b00b0 con 0x7f4af815c1c0 2024-08-15T17:00:18.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.753+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4af80b00b0 con 0x7f4af815c1c0 2024-08-15T17:00:18.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.753+0000 7f4aef7fe700 1 -- 172.21.15.146:0/1330513836 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f4af80b00b0 con 0x7f4af815c1c0 2024-08-15T17:00:18.765 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:18.766 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":21,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T16:59:43.383450+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-15T16:59:40.024316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-15T16:59:40.731449+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 msgr2=0x7f4ad8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 0x7f4ad8062d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8001690 tx=0x7f4ae8015000 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 msgr2=0x7f4af80983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4af00094d0 tx=0x7f4af000b040 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 shutdown_connections 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4ad8060890 0x7f4ad8062d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4af815c1c0 0x7f4af80983e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4af809a900 0x7f4af8097a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 --2- 172.21.15.146:0/1330513836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ae4003a60 0x7f4af8171bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 >> 172.21.15.146:0/1330513836 conn(0x7f4af80a9db0 msgr2=0x7f4af815fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 shutdown_connections 2024-08-15T17:00:18.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.757+0000 7f4afd799700 1 -- 172.21.15.146:0/1330513836 wait complete. 2024-08-15T17:00:18.780 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T17:00:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.877+0000 7f63e8bfb700 1 Processor -- start 2024-08-15T17:00:18.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.877+0000 7f63e8bfb700 1 -- start start 2024-08-15T17:00:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e406e470 0x7f63e406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63e40cad40 0x7f63e40cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e41610c0 con 0x7f63e40adf10 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e4158290 con 0x7f63e40cad40 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e415dff0 con 0x7f63e406e470 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63e40cad40 0x7f63e40cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f63e40cad40 0x7f63e40cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37764/0 (socket says 172.21.15.146:37764) 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 -- 172.21.15.146:0/4046050696 learned_addr learned my addr 172.21.15.146:0/4046050696 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63dbfff700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e406e470 0x7f63e406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 -- 172.21.15.146:0/4046050696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e406e470 msgr2=0x7f63e406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e406e470 0x7f63e406e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 -- 172.21.15.146:0/4046050696 >> v1:172.21.15.161:6789/0 conn(0x7f63e40adf10 legacy=0x7f63e40ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 -- 172.21.15.146:0/4046050696 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e415e2d0 con 0x7f63e40cad40 2024-08-15T17:00:18.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e2d9d700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40cad40 0x7f63e40cb180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009f20 tx=0x7f63dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cf731cf5df7568 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e1d9b700 1 -- 172.21.15.146:0/4046050696 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00e070 con 0x7f63e40cad40 2024-08-15T17:00:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e1d9b700 1 -- 172.21.15.146:0/4046050696 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc00b9d0 con 0x7f63e40cad40 2024-08-15T17:00:18.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e1d9b700 1 -- 172.21.15.146:0/4046050696 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0092e0 con 0x7f63e40cad40 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- 172.21.15.146:0/4046050696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40cad40 msgr2=0x7f63e40cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40cad40 0x7f63e40cb180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009f20 tx=0x7f63dc009840 comp rx=0 tx=0).stop 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- 172.21.15.146:0/4046050696 shutdown_connections 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40cad40 0x7f63e40cb180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/4046050696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e406e470 0x7f63e406e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- 172.21.15.146:0/4046050696 >> 172.21.15.146:0/4046050696 conn(0x7f63e4067810 msgr2=0x7f63e4067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- 172.21.15.146:0/4046050696 shutdown_connections 2024-08-15T17:00:18.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.881+0000 7f63e8bfb700 1 -- 172.21.15.146:0/4046050696 wait complete. 2024-08-15T17:00:18.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 Processor -- start 2024-08-15T17:00:18.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 -- start start 2024-08-15T17:00:18.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63e406e470 0x7f63e414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e40cad40 0x7f63e40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e41588b0 con 0x7f63e40adf10 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b0e20 con 0x7f63e406e470 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e8bfb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b1490 con 0x7f63e40cad40 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37774/0 (socket says 172.21.15.146:37774) 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 -- 172.21.15.146:0/3080433452 learned_addr learned my addr 172.21.15.146:0/3080433452 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e259c700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63e406e470 0x7f63e414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63e2d9d700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e40cad40 0x7f63e40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 -- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e40cad40 msgr2=0x7f63e40afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e40cad40 0x7f63e40afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 -- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63e406e470 msgr2=0x7f63e414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63e406e470 0x7f63e414c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:18.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 -- 172.21.15.146:0/3080433452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc009ae0 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.885+0000 7f63dbfff700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ecb0 tx=0x7f63cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00c850 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc00c9b0 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4158580 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00cd50 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e40b0b40 con 0x7f63e40adf10 2024-08-15T17:00:18.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4158a60 con 0x7f63e40adf10 2024-08-15T17:00:18.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f63cc013680 con 0x7f63e40adf10 2024-08-15T17:00:18.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 0x7f63c4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:18.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f63cc07cd60 con 0x7f63e40adf10 2024-08-15T17:00:18.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.889+0000 7f63e259c700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 0x7f63c4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:18.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.893+0000 7f63e259c700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 0x7f63c4062d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63d4003a80 tx=0x7f63d4007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:18.910 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:18.897+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f63e4158a60 con 0x7f63e40adf10 2024-08-15T17:00:19.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.093+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f63e4158a60 con 0x7f63e40adf10 2024-08-15T17:00:19.108 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-08-15T17:00:19.097+0000 7fbf03872700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:00:19.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.105+0000 7f63da7fc700 1 -- 172.21.15.146:0/3080433452 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f63e4158a60 con 0x7f63e40adf10 2024-08-15T17:00:19.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 msgr2=0x7f63c4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 0x7f63c4062d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63d4003a80 tx=0x7f63d4007ec0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 msgr2=0x7f63e40973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00ecb0 tx=0x7f63cc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 shutdown_connections 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f63c4060890 0x7f63c4062d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f63e40cad40 0x7f63e40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63e40adf10 0x7f63e40973d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 --2- 172.21.15.146:0/3080433452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f63e406e470 0x7f63e414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 >> 172.21.15.146:0/3080433452 conn(0x7f63e4067810 msgr2=0x7f63e40aad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 shutdown_connections 2024-08-15T17:00:19.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.109+0000 7f63e8bfb700 1 -- 172.21.15.146:0/3080433452 wait complete. 2024-08-15T17:00:19.122 INFO:tasks.ceph.mds.b.smithi161.stderr: -2255> 2024-08-15T16:59:46.837+0000 7ff6331fe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:00:19.122 INFO:teuthology.orchestra.run.smithi146.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:00:19.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T17:00:19.147 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-08-15T17:00:19.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 Processor -- start 2024-08-15T17:00:19.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 -- start start 2024-08-15T17:00:19.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6aac13fa50 0x7f6aac0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 0x7f6aac06e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac0b0810 con 0x7f6aac13f680 2024-08-15T17:00:19.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0afc70 con 0x7f6aac13fa50 2024-08-15T17:00:19.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0af060 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6aac13fa50 0x7f6aac0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6aac13fa50 0x7f6aac0aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37790/0 (socket says 172.21.15.146:37790) 2024-08-15T17:00:19.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aab7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f6aac13f680 0x7f6aac0b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45266/0 (socket says 172.21.15.146:45266) 2024-08-15T17:00:19.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 -- 172.21.15.146:0/3346888761 learned_addr learned my addr 172.21.15.146:0/3346888761 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aabfff700 1 --2- 172.21.15.146:0/3346888761 >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 0x7f6aac06e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 -- 172.21.15.146:0/3346888761 >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 msgr2=0x7f6aac06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 --2- 172.21.15.146:0/3346888761 >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 0x7f6aac06e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 -- 172.21.15.146:0/3346888761 >> v1:172.21.15.146:6789/0 conn(0x7f6aac13f680 legacy=0x7f6aac0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 -- 172.21.15.146:0/3346888761 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac0af340 con 0x7f6aac13fa50 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aabfff700 1 --2- 172.21.15.146:0/3346888761 >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 0x7f6aac06e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaaffd700 1 --2- 172.21.15.146:0/3346888761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac13fa50 0x7f6aac0aea80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6aac068ae0 tx=0x7f6a94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7dfaf0f3d5a49b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaa7fc700 1 -- 172.21.15.146:0/3346888761 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400b860 con 0x7f6aac13fa50 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaa7fc700 1 -- 172.21.15.146:0/3346888761 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9400b9c0 con 0x7f6aac13fa50 2024-08-15T17:00:19.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6aaa7fc700 1 -- 172.21.15.146:0/3346888761 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400bd60 con 0x7f6aac13fa50 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/3346888761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac13fa50 msgr2=0x7f6aac0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.233+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/3346888761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac13fa50 0x7f6aac0aea80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6aac068ae0 tx=0x7f6a94009250 comp rx=0 tx=0).stop 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/3346888761 shutdown_connections 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/3346888761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac13fa50 0x7f6aac0aea80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/3346888761 >> v2:172.21.15.161:3300/0 conn(0x7f6aac0ad6c0 0x7f6aac06e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/3346888761 >> 172.21.15.146:0/3346888761 conn(0x7f6aac0678a0 msgr2=0x7f6aac067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/3346888761 shutdown_connections 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/3346888761 wait complete. 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 Processor -- start 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- start start 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac13f680 0x7f6aac0aa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6aac13fa50 0x7f6aac0aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac0ab290 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac1589c0 con 0x7f6aac13f680 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac157db0 con 0x7f6aac13fa50 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37800/0 (socket says 172.21.15.146:37800) 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 -- 172.21.15.146:0/729123510 learned_addr learned my addr 172.21.15.146:0/729123510 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:19.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aabfff700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6aac13fa50 0x7f6aac0aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aaaffd700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac13f680 0x7f6aac0aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 -- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6aac13fa50 msgr2=0x7f6aac0aad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6aac13fa50 0x7f6aac0aad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 -- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac13f680 msgr2=0x7f6aac0aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac13f680 0x7f6aac0aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 -- 172.21.15.146:0/729123510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940094f0 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aab7fe700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400c7c0 tx=0x7f6aa4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4005a70 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4005bd0 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac158090 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40070d0 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.237+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac158650 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.241+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6aa4007440 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.241+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac064330 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.241+0000 7f6aa37fe700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.241+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f6aa407c070 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.241+0000 7f6aaaffd700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:19.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.245+0000 7f6aaaffd700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 0x7f6a90062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6aac068ae0 tx=0x7f6a94017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:19.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.249+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6aa404c3d0 con 0x7f6aac0ad6c0 2024-08-15T17:00:19.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:19.445+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6aac06ed70 con 0x7f6aac0ad6c0 2024-08-15T17:00:20.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.109+0000 7f6aa37fe700 1 -- 172.21.15.146:0/729123510 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6aa404f9f0 con 0x7f6aac0ad6c0 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 msgr2=0x7f6a90062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 0x7f6a90062d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6aac068ae0 tx=0x7f6a94017000 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 msgr2=0x7f6aac139b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400c7c0 tx=0x7f6aa4007f30 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 shutdown_connections 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6a90060860 0x7f6a90062d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6aac13fa50 0x7f6aac0aad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6aac13f680 0x7f6aac0aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 --2- 172.21.15.146:0/729123510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6aac0ad6c0 0x7f6aac139b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 >> 172.21.15.146:0/729123510 conn(0x7f6aac0678a0 msgr2=0x7f6aac13aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:20.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 shutdown_connections 2024-08-15T17:00:20.129 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.117+0000 7f6ab0a4f700 1 -- 172.21.15.146:0/729123510 wait complete. 2024-08-15T17:00:20.144 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T17:00:20.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.249+0000 7f096952a700 1 Processor -- start 2024-08-15T17:00:20.262 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.249+0000 7f096952a700 1 -- start start 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09640cd2b0 0x7f09640cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09640caeb0 0x7f09640ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09640cc480 con 0x7f09640cb300 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0964159720 con 0x7f09640cd2b0 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09640cb870 con 0x7f09640caeb0 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09640cd2b0 0x7f09640cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f09640cd2b0 0x7f09640cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37804/0 (socket says 172.21.15.146:37804) 2024-08-15T17:00:20.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 -- 172.21.15.146:0/2021240697 learned_addr learned my addr 172.21.15.146:0/2021240697 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:20.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f09627fc700 1 --2- 172.21.15.146:0/2021240697 >> v2:172.21.15.161:3300/0 conn(0x7f09640caeb0 0x7f09640ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 -- 172.21.15.146:0/2021240697 >> v2:172.21.15.161:3300/0 conn(0x7f09640caeb0 msgr2=0x7f09640ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:20.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 --2- 172.21.15.146:0/2021240697 >> v2:172.21.15.161:3300/0 conn(0x7f09640caeb0 0x7f09640ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 -- 172.21.15.146:0/2021240697 >> v1:172.21.15.161:6789/0 conn(0x7f09640cb300 legacy=0x7f09640c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 -- 172.21.15.146:0/2021240697 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640cbb50 con 0x7f09640cd2b0 2024-08-15T17:00:20.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0962ffd700 1 --2- 172.21.15.146:0/2021240697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640cd2b0 0x7f09640cd680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09640c7310 tx=0x7f094c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4a1988a27e455d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:20.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0961ffb700 1 -- 172.21.15.146:0/2021240697 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c009e70 con 0x7f09640cd2b0 2024-08-15T17:00:20.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0961ffb700 1 -- 172.21.15.146:0/2021240697 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00b860 con 0x7f09640cd2b0 2024-08-15T17:00:20.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f0961ffb700 1 -- 172.21.15.146:0/2021240697 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00bc00 con 0x7f09640cd2b0 2024-08-15T17:00:20.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- 172.21.15.146:0/2021240697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640cd2b0 msgr2=0x7f09640cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 --2- 172.21.15.146:0/2021240697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640cd2b0 0x7f09640cd680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f09640c7310 tx=0x7f094c009250 comp rx=0 tx=0).stop 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- 172.21.15.146:0/2021240697 shutdown_connections 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 --2- 172.21.15.146:0/2021240697 >> v2:172.21.15.161:3300/0 conn(0x7f09640caeb0 0x7f09640ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 --2- 172.21.15.146:0/2021240697 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640cd2b0 0x7f09640cd680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- 172.21.15.146:0/2021240697 >> 172.21.15.146:0/2021240697 conn(0x7f0964099550 msgr2=0x7f0964067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- 172.21.15.146:0/2021240697 shutdown_connections 2024-08-15T17:00:20.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.253+0000 7f096952a700 1 -- 172.21.15.146:0/2021240697 wait complete. 2024-08-15T17:00:20.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.257+0000 7f096952a700 1 Processor -- start 2024-08-15T17:00:20.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.257+0000 7f096952a700 1 -- start start 2024-08-15T17:00:20.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 0x7f096415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09640cd2b0 0x7f096415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09640c7360 con 0x7f09640caeb0 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096416a1b0 con 0x7f09640cb300 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096416a640 con 0x7f09640cd2b0 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37816/0 (socket says 172.21.15.146:37816) 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 -- 172.21.15.146:0/3246832113 learned_addr learned my addr 172.21.15.146:0/3246832113 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f09637fe700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09640cd2b0 0x7f096415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f09627fc700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 0x7f096415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 -- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09640cd2b0 msgr2=0x7f096415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09640cd2b0 0x7f096415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 -- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 msgr2=0x7f096415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 0x7f096415e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 -- 172.21.15.146:0/3246832113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0094f0 con 0x7f09640caeb0 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f09627fc700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 0x7f096415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:20.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f0962ffd700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0964088e40 tx=0x7f094c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00b9d0 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00bb30 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c025650 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0964157f60 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.261+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0964169880 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09640638f0 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f094c023070 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f095affd700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 0x7f0944062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f094c07e110 con 0x7f09640caeb0 2024-08-15T17:00:20.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f09627fc700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 0x7f0944062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:20.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.265+0000 7f09627fc700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 0x7f0944062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f09640c8770 tx=0x7f0954008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:20.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.269+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f094c04e510 con 0x7f09640caeb0 2024-08-15T17:00:20.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:20.433+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0964068020 con 0x7f09640caeb0 2024-08-15T17:00:21.128 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.117+0000 7f095affd700 1 -- 172.21.15.146:0/3246832113 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f094c051b30 con 0x7f09640caeb0 2024-08-15T17:00:21.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 msgr2=0x7f0944062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 0x7f0944062d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f09640c8770 tx=0x7f0954008040 comp rx=0 tx=0).stop 2024-08-15T17:00:21.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 msgr2=0x7f096414aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0964088e40 tx=0x7f094c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:21.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 shutdown_connections 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0944060860 0x7f0944062d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09640cd2b0 0x7f096415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09640cb300 0x7f096415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 --2- 172.21.15.146:0/3246832113 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f09640caeb0 0x7f096414aef0 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0964088e40 tx=0x7f094c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.121+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 >> 172.21.15.146:0/3246832113 conn(0x7f0964099550 msgr2=0x7f09640ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.125+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 shutdown_connections 2024-08-15T17:00:21.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.125+0000 7f096952a700 1 -- 172.21.15.146:0/3246832113 wait complete. 2024-08-15T17:00:21.137 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' removed 2024-08-15T17:00:21.149 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T17:00:21.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 Processor -- start 2024-08-15T17:00:21.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 -- start start 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 0x7f943006d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94300ca0b0 con 0x7f94300ae050 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94300c97c0 con 0x7f94300ac0d0 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94373c2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94300acb20 con 0x7f9430157050 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f9436bc1700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f94300ae050 0x7f94300ca540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45274/0 (socket says 172.21.15.146:45274) 2024-08-15T17:00:21.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f9436bc1700 1 -- 172.21.15.146:0/3927036772 learned_addr learned my addr 172.21.15.146:0/3927036772 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:21.266 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94363c0700 1 --2- 172.21.15.146:0/3927036772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 0x7f943006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111715373 0 0) 0x7f94300ca0b0 con 0x7f94300ae050 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f941c003570 con 0x7f94300ae050 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960613599 0 0) 0x7f94300c97c0 con 0x7f94300ac0d0 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94300ca0b0 con 0x7f94300ac0d0 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94363c0700 1 -- 172.21.15.146:0/3927036772 >> v1:172.21.15.161:6789/0 conn(0x7f94300ac0d0 legacy=0x7f94300ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252038750 0 0) 0x7f941c003570 con 0x7f94300ae050 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94363c0700 1 -- 172.21.15.146:0/3927036772 >> v1:172.21.15.146:6789/0 conn(0x7f94300ae050 legacy=0x7f94300ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94363c0700 1 -- 172.21.15.146:0/3927036772 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300c9a60 con 0x7f9430157050 2024-08-15T17:00:21.267 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.253+0000 7f94363c0700 1 --2- 172.21.15.146:0/3927036772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 0x7f943006d120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f942c009910 tx=0x7f942c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=808f5bc8056a2f3a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:21.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94280017f0 con 0x7f9430157050 2024-08-15T17:00:21.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c011420 con 0x7f9430157050 2024-08-15T17:00:21.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94353be700 1 -- 172.21.15.146:0/3927036772 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0117a0 con 0x7f9430157050 2024-08-15T17:00:21.268 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- 172.21.15.146:0/3927036772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 msgr2=0x7f943006d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 --2- 172.21.15.146:0/3927036772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 0x7f943006d120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f942c009910 tx=0x7f942c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- 172.21.15.146:0/3927036772 shutdown_connections 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 --2- 172.21.15.146:0/3927036772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9430157050 0x7f943006d120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- 172.21.15.146:0/3927036772 >> 172.21.15.146:0/3927036772 conn(0x7f9430066e00 msgr2=0x7f9430067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- 172.21.15.146:0/3927036772 shutdown_connections 2024-08-15T17:00:21.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- 172.21.15.146:0/3927036772 wait complete. 2024-08-15T17:00:21.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 Processor -- start 2024-08-15T17:00:21.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.257+0000 7f94373c2700 1 -- start start 2024-08-15T17:00:21.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94300ac0d0 0x7f94300a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:21.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:21.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9430157050 0x7f94300a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:21.273 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300a5ab0 con 0x7f94300ae050 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94300a9f80 con 0x7f9430157050 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94300aa200 con 0x7f94300ac0d0 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37828/0 (socket says 172.21.15.146:37828) 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 -- 172.21.15.146:0/192341499 learned_addr learned my addr 172.21.15.146:0/192341499 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9436bc1700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9430157050 0x7f94300a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94363c0700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94300ac0d0 0x7f94300a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 -- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94300ac0d0 msgr2=0x7f94300a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94300ac0d0 0x7f94300a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 -- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9430157050 msgr2=0x7f94300a7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9430157050 0x7f94300a7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 -- 172.21.15.146:0/192341499 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009420 con 0x7f94300ae050 2024-08-15T17:00:21.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f9435bbf700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f942000c580 tx=0x7f9420007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:21.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420010510 con 0x7f94300ae050 2024-08-15T17:00:21.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9420010670 con 0x7f94300ae050 2024-08-15T17:00:21.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94300aa4e0 con 0x7f94300ae050 2024-08-15T17:00:21.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420010a30 con 0x7f94300ae050 2024-08-15T17:00:21.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.261+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94300aaaa0 con 0x7f94300ae050 2024-08-15T17:00:21.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94300aacc0 con 0x7f94300ae050 2024-08-15T17:00:21.276 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f942001b400 con 0x7f94300ae050 2024-08-15T17:00:21.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94267fc700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 0x7f9414062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:21.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f942007c640 con 0x7f94300ae050 2024-08-15T17:00:21.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94363c0700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 0x7f9414062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:21.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.265+0000 7f94363c0700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 0x7f9414062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f942c0091e0 tx=0x7f942c016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:21.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.273+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f942004cbf0 con 0x7f94300ae050 2024-08-15T17:00:21.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:21.441+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f943006d5a0 con 0x7f94300ae050 2024-08-15T17:00:22.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.129+0000 7f94267fc700 1 -- 172.21.15.146:0/192341499 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9420050210 con 0x7f94300ae050 2024-08-15T17:00:22.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 msgr2=0x7f9414062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 0x7f9414062d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f942c0091e0 tx=0x7f942c016000 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 msgr2=0x7f9430140b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f942000c580 tx=0x7f9420007e10 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 shutdown_connections 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9414060860 0x7f9414062d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9430157050 0x7f94300a7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f94300ae050 0x7f9430140b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 --2- 172.21.15.146:0/192341499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94300ac0d0 0x7f94300a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 >> 172.21.15.146:0/192341499 conn(0x7f9430066e00 msgr2=0x7f94300ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 shutdown_connections 2024-08-15T17:00:22.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.133+0000 7f94373c2700 1 -- 172.21.15.146:0/192341499 wait complete. 2024-08-15T17:00:22.147 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' removed 2024-08-15T17:00:22.159 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-15T17:00:22.159 INFO:tasks.ceph.mds.a:Stopping old one... 2024-08-15T17:00:22.159 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T17:00:22.159 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:00:22.159 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-15T17:00:22.159 INFO:tasks.ceph.mds.c:Stopping old one... 2024-08-15T17:00:22.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T17:00:22.160 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:00:22.160 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-15T17:00:22.160 INFO:tasks.ceph.mds.b:Stopping old one... 2024-08-15T17:00:22.160 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T17:00:22.160 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:00:22.161 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-15T17:00:22.161 INFO:tasks.ceph.mds.d:Stopping old one... 2024-08-15T17:00:22.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T17:00:22.161 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:00:22.171 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T17:00:22.171 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-15T17:00:22.172 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T17:00:22.172 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-15T17:00:22.173 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T17:00:22.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-15T17:00:22.174 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T17:00:22.175 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-15T17:00:22.175 INFO:tasks.ceph.mds.b:Started 2024-08-15T17:00:22.176 DEBUG:teuthology.parallel:result is None 2024-08-15T17:00:22.177 INFO:tasks.ceph.mds.d:Started 2024-08-15T17:00:22.177 DEBUG:teuthology.parallel:result is None 2024-08-15T17:00:22.217 INFO:tasks.ceph.mds.c:Started 2024-08-15T17:00:22.217 DEBUG:teuthology.parallel:result is None 2024-08-15T17:00:22.218 INFO:tasks.ceph.mds.a:Started 2024-08-15T17:00:22.218 DEBUG:teuthology.parallel:result is None 2024-08-15T17:00:22.218 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-08-15T17:00:22.242 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-08-15T17:00:22.244 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-08-15T17:00:22.300 INFO:tasks.ceph.mds.a.smithi146.stdout:starting mds.a at 2024-08-15T17:00:22.301 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-08-15T17:00:22.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 Processor -- start 2024-08-15T17:00:22.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 -- start start 2024-08-15T17:00:22.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fea000cbb60 0x7fea000b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea000ca3b0 con 0x7fea000c8720 2024-08-15T17:00:22.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea000ca690 con 0x7fea0015a200 2024-08-15T17:00:22.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06e13700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea000ca970 con 0x7fea000cbb60 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05610700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fea000c8720 0x7fea000cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45288/0 (socket says 172.21.15.146:45288) 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05610700 1 -- 172.21.15.146:0/14036013 learned_addr learned my addr 172.21.15.146:0/14036013 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 --2- 172.21.15.146:0/14036013 >> v2:172.21.15.161:3300/0 conn(0x7fea000cbb60 0x7fea000b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662801396 0 0) 0x7fea000ca3b0 con 0x7fea000c8720 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ec003570 con 0x7fea000c8720 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422394670 0 0) 0x7fe9ec003570 con 0x7fea000c8720 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 >> v2:172.21.15.161:3300/0 conn(0x7fea000cbb60 msgr2=0x7fea000b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 --2- 172.21.15.146:0/14036013 >> v2:172.21.15.161:3300/0 conn(0x7fea000cbb60 0x7fea000b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 >> v1:172.21.15.161:6789/0 conn(0x7fea0015a200 legacy=0x7fea000abe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000cac50 con 0x7fea000c8720 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7fe9fc001470 con 0x7fea000c8720 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 >> v1:172.21.15.146:6789/0 conn(0x7fea000c8720 legacy=0x7fea000cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9ec003a60 0x7fe9ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.325 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ec004810 0x7fe9ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fe9ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000ca970 con 0x7fe9ec0051c0 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec00c0d0 con 0x7fe9ec004810 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec00c290 con 0x7fe9ec003a60 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fe9ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05610700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ec004810 0x7fe9ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05e11700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9ec003a60 0x7fe9ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 -- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9ec003a60 msgr2=0x7fe9ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9ec003a60 0x7fe9ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 -- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ec004810 msgr2=0x7fe9ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ec004810 0x7fe9ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea06612700 1 -- 172.21.15.146:0/14036013 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec003570 con 0x7fe9ec0051c0 2024-08-15T17:00:22.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05e11700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe9ec003a60 0x7fe9ec003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.309+0000 7fea05610700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9ec004810 0x7fe9ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06612700 1 -- 172.21.15.146:0/14036013 reap_dead start 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06612700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fe9ec00ba60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e210 tx=0x7fe9f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e89aec0683d4f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800eac0 con 0x7fe9ec0051c0 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f800f040 con 0x7fe9ec0051c0 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea04e0f700 1 -- 172.21.15.146:0/14036013 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f800ee70 con 0x7fe9ec0051c0 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 msgr2=0x7fe9ec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fe9ec00ba60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e210 tx=0x7fe9f800b040 comp rx=0 tx=0).stop 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- 172.21.15.146:0/14036013 shutdown_connections 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 --2- 172.21.15.146:0/14036013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fe9ec00ba60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- 172.21.15.146:0/14036013 >> 172.21.15.146:0/14036013 conn(0x7fea0006d800 msgr2=0x7fea0006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- 172.21.15.146:0/14036013 shutdown_connections 2024-08-15T17:00:22.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- 172.21.15.146:0/14036013 wait complete. 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 Processor -- start 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- start start 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea00144eb0 0x7fea00149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0014b350 0x7fea0014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.330 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00159a30 con 0x7fe9ec0051c0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00159060 con 0x7fea00144eb0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea06e13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea001491a0 con 0x7fea0014b350 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.313+0000 7fea05e11700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37866/0 (socket says 172.21.15.146:37866) 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 -- 172.21.15.146:0/3466012909 learned_addr learned my addr 172.21.15.146:0/3466012909 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05610700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea00144eb0 0x7fea00149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea06612700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0014b350 0x7fea0014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 -- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0014b350 msgr2=0x7fea0014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0014b350 0x7fea0014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 -- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea00144eb0 msgr2=0x7fea00149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea00144eb0 0x7fea00149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 -- 172.21.15.146:0/3466012909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00158a50 con 0x7fe9ec0051c0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05e11700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e580 tx=0x7fe9f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:22.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400ee50 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4010040 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4011670 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea00148940 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea00145320 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe9f40119e0 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea000cb170 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 0x7fe9e4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe9f407b970 con 0x7fe9ec0051c0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.317+0000 7fea05610700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 0x7fe9e4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:22.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.321+0000 7fea05610700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 0x7fe9e4062d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000ff0 tx=0x7fe9fc007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:22.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.325+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fea000cb170 con 0x7fe9ec0051c0 2024-08-15T17:00:22.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:22.505+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fea000cb170 con 0x7fe9ec0051c0 2024-08-15T17:00:23.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.145+0000 7fe9f1ffb700 1 -- 172.21.15.146:0/3466012909 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea000cb170 con 0x7fe9ec0051c0 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 msgr2=0x7fe9e4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 0x7fe9e4062d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc000ff0 tx=0x7fe9fc007040 comp rx=0 tx=0).stop 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 msgr2=0x7fea001594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e580 tx=0x7fe9f4009040 comp rx=0 tx=0).stop 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 shutdown_connections 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060890 0x7fe9e4062d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea0014b350 0x7fea0014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea00144eb0 0x7fea00149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 --2- 172.21.15.146:0/3466012909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe9ec0051c0 0x7fea001594f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 >> 172.21.15.146:0/3466012909 conn(0x7fea0006d800 msgr2=0x7fea001570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:23.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 shutdown_connections 2024-08-15T17:00:23.164 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.149+0000 7fea06e13700 1 -- 172.21.15.146:0/3466012909 wait complete. 2024-08-15T17:00:23.164 INFO:teuthology.orchestra.run.smithi146.stderr: removed all blocklist entries 2024-08-15T17:00:23.181 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-08-15T17:00:23.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.265+0000 7f220b59e700 1 Processor -- start 2024-08-15T17:00:23.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.265+0000 7f220b59e700 1 -- start start 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 0x7f220c160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220c156910 con 0x7f220c0acfb0 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220b59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220c1565e0 con 0x7f220c155c60 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c0cc3d0 con 0x7f220c068db0 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2203fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f220c0acfb0 0x7f220c0686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45296/0 (socket says 172.21.15.146:45296) 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2203fff700 1 -- 172.21.15.146:0/1729783005 learned_addr learned my addr 172.21.15.146:0/1729783005 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:23.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220ad9d700 1 --2- 172.21.15.146:0/1729783005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 0x7f220c160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664521533 0 0) 0x7f220c156910 con 0x7f220c0acfb0 2024-08-15T17:00:23.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f8003570 con 0x7f220c0acfb0 2024-08-15T17:00:23.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204491097 0 0) 0x7f220c1565e0 con 0x7f220c155c60 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f220c156910 con 0x7f220c155c60 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220ad9d700 1 -- 172.21.15.146:0/1729783005 >> v1:172.21.15.161:6789/0 conn(0x7f220c155c60 legacy=0x7f220c06b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220ad9d700 1 -- 172.21.15.146:0/1729783005 >> v1:172.21.15.146:6789/0 conn(0x7f220c0acfb0 legacy=0x7f220c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220ad9d700 1 -- 172.21.15.146:0/1729783005 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0ad470 con 0x7f220c068db0 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f220ad9d700 1 --2- 172.21.15.146:0/1729783005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 0x7f220c160d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e90 tx=0x7f2204009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d198fc3fdf74758 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400e070 con 0x7f220c068db0 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220400bb80 con 0x7f220c068db0 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.269+0000 7f2209d9b700 1 -- 172.21.15.146:0/1729783005 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204012430 con 0x7f220c068db0 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/1729783005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 msgr2=0x7f220c160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 --2- 172.21.15.146:0/1729783005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 0x7f220c160d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2204009e90 tx=0x7f2204009780 comp rx=0 tx=0).stop 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/1729783005 shutdown_connections 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 --2- 172.21.15.146:0/1729783005 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c068db0 0x7f220c160d90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/1729783005 >> 172.21.15.146:0/1729783005 conn(0x7f220c06d820 msgr2=0x7f220c06dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:23.284 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/1729783005 shutdown_connections 2024-08-15T17:00:23.285 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/1729783005 wait complete. 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 Processor -- start 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- start start 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f220c0acfb0 0x7f220c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c155c60 0x7f220c14a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c14f6b0 con 0x7f220c068db0 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c14f0f0 con 0x7f220c0acfb0 2024-08-15T17:00:23.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220c14f3d0 con 0x7f220c155c60 2024-08-15T17:00:23.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220ad9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c155c60 0x7f220c14a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37878/0 (socket says 172.21.15.146:37878) 2024-08-15T17:00:23.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 -- 172.21.15.146:0/2811715652 learned_addr learned my addr 172.21.15.146:0/2811715652 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:23.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f2203fff700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f220c0acfb0 0x7f220c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 -- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c155c60 msgr2=0x7f220c14a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c155c60 0x7f220c14a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 -- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f220c0acfb0 msgr2=0x7f220c0a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f220c0acfb0 0x7f220c0a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 -- 172.21.15.146:0/2811715652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204009a20 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220a59c700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c580 tx=0x7f21fc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc010510 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc010670 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc010a30 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.273+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c151610 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c0a5990 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c0638f0 con 0x7f220c068db0 2024-08-15T17:00:23.289 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f21fc01b400 con 0x7f220c068db0 2024-08-15T17:00:23.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f22027fc700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 0x7f21ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f21fc07c380 con 0x7f220c068db0 2024-08-15T17:00:23.290 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.277+0000 7f2203fff700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 0x7f21ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.291 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.281+0000 7f2203fff700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 0x7f21ec062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f220c1549e0 tx=0x7f21f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.299 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.285+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f21fc04cb20 con 0x7f220c068db0 2024-08-15T17:00:23.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.509+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f220c14f7f0 con 0x7f220c068db0 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.509+0000 7f22027fc700 1 -- 172.21.15.146:0/2811715652 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f21fc050140 con 0x7f220c068db0 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: "auth_dump": [ 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mds.a", 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75maQcKDRAAWdaN3na/L2qB437SOQKc5w==", 2024-08-15T17:00:23.521 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow *", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow *", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mds.b", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mHUC7FhAAq9gwtEFOiyqrnPvAeu9Oqw==", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.522 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow *", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow *", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mds.c", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mMlx3EBAA7H9f6HWTvB44NXUPLs9bjQ==", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow *", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow *", 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.523 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mds.d", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mcN3qHBAA5fqtrfo5CP4R9JrI4toBPg==", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow *", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow *", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.0", 2024-08-15T17:00:23.524 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDLM75mfEcDExAAAwHP4sfPsT0ZgUZ7AHH5PA==", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.1", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDOM75mOvtwGRAAlfIQF/O9oyllcVWZTeOlwg==", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.525 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.2", 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDRM75mOyI4HBAAp8V4uXtcNax/1vNYkp9XTA==", 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.526 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.3", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDUM75m9kjbIRAAhlj6FuHE+AvOaNzV/38Q+g==", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.4", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDaM75m4xx/AhAA2ZCxJiS34e6Mpappwdir4A==", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.527 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.5", 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDdM75mZ5a6BhAAs6AtJ5GJlE49U9sobLhKgQ==", 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.528 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.6", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDgM75mCf1eCRAAGDg04874V5e2DzHUNM0Q5A==", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "osd.7", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDjM75mC8QmDhAAPpGC18CrTOzLgairACuZOg==", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow profile osd", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile osd", 2024-08-15T17:00:23.529 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.0", 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mEyclIBAAjIGu9uX2cilmZc1BD1bFbA==", 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow r", 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow rw", 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow rwx" 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.530 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.1", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75m7PlkJRAAnPA93Ew9D+wE4HbBHV98Zg==", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow r", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow rw", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow rwx" 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.2", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mwQPFIhAAy72vH4sSpT+chLV9GuGthg==", 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.531 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow r", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow rw", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow rwx" 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.3", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mSRn9KhAAcY7ez+KrfzxZ/w00NzSg+Q==", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow r", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow rw", 2024-08-15T17:00:23.532 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow rwx" 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.admin", 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDHM75mPQxZNhAAuOM7kcvuPiOJsTt59K0nBw==", 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow *", 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "mgr": "allow *", 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow *", 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.533 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-mds", 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75mokpDExAAo9EFXkSfk/Pok2TEEcEU8w==", 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-mds" 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-mgr", 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75mOWZDExAAer94Jbb+tqH2ZeB4nFbzNQ==", 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-mgr" 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.534 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-osd", 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75mAYBDExAAf3ykCLR5Rqia+kTf7D5T+g==", 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-osd" 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-rbd", 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75m5JlDExAASPfd4DgtgRUK5iojrjAsSA==", 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-rbd" 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.535 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75m+rBDExAA+na2LCDjl/pTjw5rBE4V1Q==", 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "client.bootstrap-rgw", 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDtM75mlchDExAAiKVjojM86rwinr0s1mJtTw==", 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile bootstrap-rgw" 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.536 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mgr.x", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75mCt0hChAApiDTDwbfZMW6wR3FSTc0cg==", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow *", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile mgr", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "entity": "mgr.y", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "key": "AQDIM75msKqIBxAA4N3PyxiT1fLJrG8xeSHh0g==", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": { 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": "allow *", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "mon": "allow profile mgr", 2024-08-15T17:00:23.537 INFO:teuthology.orchestra.run.smithi146.stdout: "osd": "allow *" 2024-08-15T17:00:23.538 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.538 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:23.538 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-08-15T17:00:23.538 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:00:23.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 msgr2=0x7f21ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 0x7f21ec062d10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f220c1549e0 tx=0x7f21f4009040 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 msgr2=0x7f220c14a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c580 tx=0x7f21fc007e10 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 shutdown_connections 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f21ec060860 0x7f21ec062d10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f220c155c60 0x7f220c14a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f220c0acfb0 0x7f220c0a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 --2- 172.21.15.146:0/2811715652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f220c068db0 0x7f220c14a9e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 >> 172.21.15.146:0/2811715652 conn(0x7f220c06d820 msgr2=0x7f220c159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 shutdown_connections 2024-08-15T17:00:23.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.513+0000 7f220b59e700 1 -- 172.21.15.146:0/2811715652 wait complete. 2024-08-15T17:00:23.541 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-08-15T17:00:23.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 Processor -- start 2024-08-15T17:00:23.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 -- start start 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f280d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f280cf190 con 0x7f7f28159040 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f280d09a0 con 0x7f7f280d0010 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2d593700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280ce5e0 con 0x7f7f2815d5b0 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f27fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7f28159040 0x7f7f280cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45308/0 (socket says 172.21.15.146:45308) 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f27fff700 1 -- 172.21.15.146:0/486891473 learned_addr learned my addr 172.21.15.146:0/486891473 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f2cd92700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f280d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129988176 0 0) 0x7f7f280cf190 con 0x7f7f28159040 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f10003570 con 0x7f7f28159040 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 41607924 0 0) 0x7f7f10003570 con 0x7f7f28159040 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 msgr2=0x7f7f280d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f280d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 >> v1:172.21.15.161:6789/0 conn(0x7f7f280d0010 legacy=0x7f7f2815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f280ce860 con 0x7f7f28159040 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f7f18001470 con 0x7f7f28159040 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 >> v1:172.21.15.146:6789/0 conn(0x7f7f28159040 legacy=0x7f7f280cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f10003a90 0x7f7f10003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f10005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.650 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280ce5e0 con 0x7f7f10004810 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1000bc20 con 0x7f7f10003a90 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.633+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1000bde0 con 0x7f7f2815d5b0 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f27fff700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f10003a90 0x7f7f10003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2cd92700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f10005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 -- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 msgr2=0x7f7f10005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f2815d5b0 0x7f7f10005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 -- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f10003a90 msgr2=0x7f7f10003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f10003a90 0x7f7f10003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 -- 172.21.15.146:0/486891473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f10003570 con 0x7f7f10004810 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f27fff700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f10003a90 0x7f7f10003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f277fe700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f10004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c001880 tx=0x7f7f1c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1245ffd49a260552 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c012420 con 0x7f7f10004810 2024-08-15T17:00:23.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c012580 con 0x7f7f10004810 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f26ffd700 1 -- 172.21.15.146:0/486891473 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c012920 con 0x7f7f10004810 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- 172.21.15.146:0/486891473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 msgr2=0x7f7f10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 --2- 172.21.15.146:0/486891473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f10004c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c001880 tx=0x7f7f1c007d30 comp rx=0 tx=0).stop 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2cd92700 1 -- 172.21.15.146:0/486891473 reap_dead start 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- 172.21.15.146:0/486891473 shutdown_connections 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- 172.21.15.146:0/486891473 >> 172.21.15.146:0/486891473 conn(0x7f7f28068250 msgr2=0x7f7f28068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- 172.21.15.146:0/486891473 shutdown_connections 2024-08-15T17:00:23.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- 172.21.15.146:0/486891473 wait complete. 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 Processor -- start 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.637+0000 7f7f2d593700 1 -- start start 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 0x7f7f280b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f28149b20 0x7f7f280b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2814bec0 con 0x7f7f10004810 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f280afb50 con 0x7f7f2814ba50 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2816d360 con 0x7f7f28149b20 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37894/0 (socket says 172.21.15.146:37894) 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 -- 172.21.15.146:0/136071769 learned_addr learned my addr 172.21.15.146:0/136071769 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2cd92700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f28149b20 0x7f7f280b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f277fe700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 0x7f7f280b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 -- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f28149b20 msgr2=0x7f7f280b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f28149b20 0x7f7f280b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 -- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 msgr2=0x7f7f280b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 0x7f7f280b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 -- 172.21.15.146:0/136071769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2816d640 con 0x7f7f10004810 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f277fe700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 0x7f7f280b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f27fff700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800eb40 tx=0x7f7f180010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18005e70 con 0x7f7f10004810 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1800c7e0 con 0x7f7f10004810 2024-08-15T17:00:23.656 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800cb80 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2816d920 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2816dc00 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f280643a0 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7f1801d460 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 0x7f7f14062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7f1807c7e0 con 0x7f7f10004810 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.641+0000 7f7f277fe700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 0x7f7f14062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:23.657 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.645+0000 7f7f277fe700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 0x7f7f14062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f281533e0 tx=0x7f7f1c015000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:23.661 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.649+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7f18050a10 con 0x7f7f10004810 2024-08-15T17:00:23.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.873+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7f280b3480 con 0x7f7f10004810 2024-08-15T17:00:23.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f0ffff700 1 -- 172.21.15.146:0/136071769 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7f180505a0 con 0x7f7f10004810 2024-08-15T17:00:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 msgr2=0x7f7f14062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 0x7f7f14062d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7f281533e0 tx=0x7f7f1c015000 comp rx=0 tx=0).stop 2024-08-15T17:00:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 msgr2=0x7f7f2816cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:23.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800eb40 tx=0x7f7f180010f0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 shutdown_connections 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7f14060860 0x7f7f14062d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f28149b20 0x7f7f280b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7f2814ba50 0x7f7f280b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 --2- 172.21.15.146:0/136071769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f10004810 0x7f7f2816cd80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 >> 172.21.15.146:0/136071769 conn(0x7f7f28068250 msgr2=0x7f7f281580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.881+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 shutdown_connections 2024-08-15T17:00:23.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:23.885+0000 7f7f2d593700 1 -- 172.21.15.146:0/136071769 wait complete. 2024-08-15T17:00:23.909 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-08-15T17:00:24.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.009+0000 7fc79b7cf700 1 Processor -- start 2024-08-15T17:00:24.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.009+0000 7fc79b7cf700 1 -- start start 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc79415a620 0x7fc7940ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7940ae2a0 0x7fc7940a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940a9160 con 0x7fc7940b0f10 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940aff10 con 0x7fc7940ae2a0 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940b1680 con 0x7fc79415a620 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc7940ae2a0 0x7fc7940a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc798d6a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc7940b0f10 0x7fc7940adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45314/0 (socket says 172.21.15.146:45314) 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc798d6a700 1 -- 172.21.15.146:0/2045725987 learned_addr learned my addr 172.21.15.146:0/2045725987 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:24.024 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79956b700 1 --2- 172.21.15.146:0/2045725987 >> v2:172.21.15.161:3300/0 conn(0x7fc79415a620 0x7fc7940ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.025 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc793fff700 1 -- 172.21.15.146:0/2045725987 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115228740 0 0) 0x7fc7940a9160 con 0x7fc7940b0f10 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 -- 172.21.15.146:0/2045725987 >> v2:172.21.15.161:3300/0 conn(0x7fc79415a620 msgr2=0x7fc7940ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 --2- 172.21.15.146:0/2045725987 >> v2:172.21.15.161:3300/0 conn(0x7fc79415a620 0x7fc7940ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 -- 172.21.15.146:0/2045725987 >> v1:172.21.15.146:6789/0 conn(0x7fc7940b0f10 legacy=0x7fc7940adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 -- 172.21.15.146:0/2045725987 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940b1960 con 0x7fc7940ae2a0 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc799d6c700 1 --2- 172.21.15.146:0/2045725987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940ae2a0 0x7fc7940a95f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc79406b410 tx=0x7fc78c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0078d8ebfb4d42 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc793fff700 1 -- 172.21.15.146:0/2045725987 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c00e070 con 0x7fc7940ae2a0 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc793fff700 1 -- 172.21.15.146:0/2045725987 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c00bb60 con 0x7fc7940ae2a0 2024-08-15T17:00:24.026 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc793fff700 1 -- 172.21.15.146:0/2045725987 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c012430 con 0x7fc7940ae2a0 2024-08-15T17:00:24.027 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2045725987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940ae2a0 msgr2=0x7fc7940a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2045725987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940ae2a0 0x7fc7940a95f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc79406b410 tx=0x7fc78c009780 comp rx=0 tx=0).stop 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2045725987 shutdown_connections 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2045725987 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940ae2a0 0x7fc7940a95f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2045725987 >> v2:172.21.15.161:3300/0 conn(0x7fc79415a620 0x7fc7940ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2045725987 >> 172.21.15.146:0/2045725987 conn(0x7fc794067810 msgr2=0x7fc794067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2045725987 shutdown_connections 2024-08-15T17:00:24.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.013+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2045725987 wait complete. 2024-08-15T17:00:24.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 Processor -- start 2024-08-15T17:00:24.029 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 -- start start 2024-08-15T17:00:24.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7940ae2a0 0x7fc7940973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.031 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc79415a620 0x7fc7940c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794167200 con 0x7fc7940b0f10 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794097c10 con 0x7fc7940ae2a0 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79b7cf700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc794097ef0 con 0x7fc79415a620 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37912/0 (socket says 172.21.15.146:37912) 2024-08-15T17:00:24.032 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 -- 172.21.15.146:0/2567977109 learned_addr learned my addr 172.21.15.146:0/2567977109 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc79956b700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7940ae2a0 0x7fc7940973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc799d6c700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc79415a620 0x7fc7940c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 -- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc79415a620 msgr2=0x7fc7940c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc79415a620 0x7fc7940c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 -- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7940ae2a0 msgr2=0x7fc7940973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7940ae2a0 0x7fc7940973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 -- 172.21.15.146:0/2567977109 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c009a20 con 0x7fc7940b0f10 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.017+0000 7fc798d6a700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c7e0 tx=0x7fc784007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784005b30 con 0x7fc7940b0f10 2024-08-15T17:00:24.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784005c90 con 0x7fc7940b0f10 2024-08-15T17:00:24.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7940981d0 con 0x7fc7940b0f10 2024-08-15T17:00:24.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784007300 con 0x7fc7940b0f10 2024-08-15T17:00:24.034 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940978f0 con 0x7fc7940b0f10 2024-08-15T17:00:24.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc794167370 con 0x7fc7940b0f10 2024-08-15T17:00:24.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc7840076b0 con 0x7fc7940b0f10 2024-08-15T17:00:24.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc78407bf20 con 0x7fc7940b0f10 2024-08-15T17:00:24.036 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.021+0000 7fc79956b700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.025+0000 7fc79956b700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 0x7fc774062d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d10 tx=0x7fc77c0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.025+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc78404c6c0 con 0x7fc7940b0f10 2024-08-15T17:00:24.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.245+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc7940645c0 con 0x7fc7940b0f10 2024-08-15T17:00:24.265 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.253+0000 7fc7917fa700 1 -- 172.21.15.146:0/2567977109 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc7840170a0 con 0x7fc7940b0f10 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 msgr2=0x7fc774062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 0x7fc774062d10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d10 tx=0x7fc77c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 msgr2=0x7fc794156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c7e0 tx=0x7fc784007df0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 shutdown_connections 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc774060860 0x7fc774062d10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc79415a620 0x7fc7940c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc7940b0f10 0x7fc794156860 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 --2- 172.21.15.146:0/2567977109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc7940ae2a0 0x7fc7940973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 >> 172.21.15.146:0/2567977109 conn(0x7fc794067810 msgr2=0x7fc7940a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:24.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 shutdown_connections 2024-08-15T17:00:24.270 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.257+0000 7fc79b7cf700 1 -- 172.21.15.146:0/2567977109 wait complete. 2024-08-15T17:00:24.281 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-08-15T17:00:24.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.373+0000 7f4aaffff700 1 Processor -- start 2024-08-15T17:00:24.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.373+0000 7f4aaffff700 1 -- start start 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.373+0000 7f4aaffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4ab0068ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00cafc0 con 0x7f4ab00caa20 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00cb2a0 con 0x7f4ab00cc060 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00cb580 con 0x7f4ab00688d0 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f4ab00caa20 0x7f4ab0156800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45318/0 (socket says 172.21.15.146:45318) 2024-08-15T17:00:24.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/443073351 learned_addr learned my addr 172.21.15.146:0/443073351 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaeffd700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4ab0068ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635591793 0 0) 0x7f4ab00cafc0 con 0x7f4ab00caa20 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a90003570 con 0x7f4ab00caa20 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1301474010 0 0) 0x7f4a90003570 con 0x7f4ab00caa20 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 msgr2=0x7f4ab0068ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4ab0068ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 >> v1:172.21.15.161:6789/0 conn(0x7f4ab00cc060 legacy=0x7f4ab00b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00cb860 con 0x7f4ab00caa20 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f4aa80017f0 con 0x7f4ab00caa20 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 >> v1:172.21.15.146:6789/0 conn(0x7f4ab00caa20 legacy=0x7f4ab0156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4a90003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a90004010 0x7f4a90004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a90004d50 0x7f4a900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00cb580 con 0x7f4a90004d50 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9000bbf0 con 0x7f4a90004010 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9000bdb0 con 0x7f4ab00688d0 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a90004d50 0x7f4a900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aa7fff700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a90004010 0x7f4a90004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaeffd700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4a90003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 msgr2=0x7f4a90003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ab00688d0 0x7f4a90003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a90004010 msgr2=0x7f4a90004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a90004010 0x7f4a90004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/443073351 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90003570 con 0x7f4a90004d50 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aa7fff700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a90004010 0x7f4a90004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a90004d50 0x7f4a900051c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800a900 tx=0x7f4aa80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8b1ebf66f3a23b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80070d0 con 0x7f4a90004d50 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8007230 con 0x7f4a90004d50 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aae7fc700 1 -- 172.21.15.146:0/443073351 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80075f0 con 0x7f4a90004d50 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- 172.21.15.146:0/443073351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a90004d50 msgr2=0x7f4a900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 --2- 172.21.15.146:0/443073351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a90004d50 0x7f4a900051c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800a900 tx=0x7f4aa80021d0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/443073351 reap_dead start 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- 172.21.15.146:0/443073351 shutdown_connections 2024-08-15T17:00:24.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- 172.21.15.146:0/443073351 >> 172.21.15.146:0/443073351 conn(0x7f4ab006d790 msgr2=0x7f4ab006db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:24.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- 172.21.15.146:0/443073351 shutdown_connections 2024-08-15T17:00:24.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.377+0000 7f4aaffff700 1 -- 172.21.15.146:0/443073351 wait complete. 2024-08-15T17:00:24.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.381+0000 7f4aaffff700 1 Processor -- start 2024-08-15T17:00:24.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.381+0000 7f4aaffff700 1 -- start start 2024-08-15T17:00:24.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a90004d50 0x7f4ab014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ab0169e80 0x7f4ab00aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00ab3b0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00ab110 con 0x7f4ab0169e80 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab016a2f0 con 0x7f4a90004d50 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37930/0 (socket says 172.21.15.146:37930) 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/3349888827 learned_addr learned my addr 172.21.15.146:0/3349888827 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaeffd700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a90004d50 0x7f4ab014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aa7fff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ab0169e80 0x7f4ab00aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a90004d50 msgr2=0x7f4ab014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a90004d50 0x7f4ab014cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ab0169e80 msgr2=0x7f4ab00aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ab0169e80 0x7f4ab00aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 -- 172.21.15.146:0/3349888827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00af710 con 0x7f4ab00c68f0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaf7fe700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8012000 tx=0x7f4aa8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80073a0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8007500 con 0x7f4ab00c68f0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab00af9f0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa801e680 con 0x7f4ab00c68f0 2024-08-15T17:00:24.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab00affb0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.385+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab00b01a0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.389+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f4aa801ea30 con 0x7f4ab00c68f0 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.389+0000 7f4aa6ffd700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 0x7f4a94062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.389+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4aa807e0f0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.389+0000 7f4aaeffd700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 0x7f4a94062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.389+0000 7f4aaeffd700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 0x7f4a94062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009db0 tx=0x7f4aa0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.404 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.393+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f4aa804e940 con 0x7f4ab00c68f0 2024-08-15T17:00:24.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.613+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4ab00643a0 con 0x7f4ab00c68f0 2024-08-15T17:00:24.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.621+0000 7f4aa6ffd700 1 -- 172.21.15.146:0/3349888827 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4aa8051f60 con 0x7f4ab00c68f0 2024-08-15T17:00:24.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 msgr2=0x7f4a94062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 0x7f4a94062d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009db0 tx=0x7f4aa0008040 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 msgr2=0x7f4ab00a8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8012000 tx=0x7f4aa8016000 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 shutdown_connections 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4a94060860 0x7f4a94062d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4ab00c68f0 0x7f4ab00a8d60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ab0169e80 0x7f4ab00aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 --2- 172.21.15.146:0/3349888827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a90004d50 0x7f4ab014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 >> 172.21.15.146:0/3349888827 conn(0x7f4ab006d790 msgr2=0x7f4ab00ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 shutdown_connections 2024-08-15T17:00:24.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.625+0000 7f4aaffff700 1 -- 172.21.15.146:0/3349888827 wait complete. 2024-08-15T17:00:24.649 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-08-15T17:00:24.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 Processor -- start 2024-08-15T17:00:24.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 -- start start 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f078c0d0010 0x7f078c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f078c15cea0 0x7f078c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0cf190 con 0x7f078c159040 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c0d09a0 con 0x7f078c15cea0 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078b59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c0ce5e0 con 0x7f078c0d0010 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f078c15cea0 0x7f078c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f078c159040 0x7f078c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45322/0 (socket says 172.21.15.146:45322) 2024-08-15T17:00:24.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078a59c700 1 -- 172.21.15.146:0/238606503 learned_addr learned my addr 172.21.15.146:0/238606503 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:24.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f0789d9b700 1 --2- 172.21.15.146:0/238606503 >> v2:172.21.15.161:3300/0 conn(0x7f078c0d0010 0x7f078c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078959a700 1 -- 172.21.15.146:0/238606503 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175995880 0 0) 0x7f078c0cf190 con 0x7f078c159040 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078959a700 1 -- 172.21.15.146:0/238606503 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774003570 con 0x7f078c159040 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 -- 172.21.15.146:0/238606503 >> v2:172.21.15.161:3300/0 conn(0x7f078c0d0010 msgr2=0x7f078c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 --2- 172.21.15.146:0/238606503 >> v2:172.21.15.161:3300/0 conn(0x7f078c0d0010 0x7f078c0d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 -- 172.21.15.146:0/238606503 >> v1:172.21.15.146:6789/0 conn(0x7f078c159040 legacy=0x7f078c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 -- 172.21.15.146:0/238606503 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0ce860 con 0x7f078c15cea0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078ad9d700 1 --2- 172.21.15.146:0/238606503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c15cea0 0x7f078c15d2e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f078c067ef0 tx=0x7f0784009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f36d9ae797cf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078959a700 1 -- 172.21.15.146:0/238606503 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400e070 con 0x7f078c15cea0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078959a700 1 -- 172.21.15.146:0/238606503 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078400bab0 con 0x7f078c15cea0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.749+0000 7f078959a700 1 -- 172.21.15.146:0/238606503 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400be50 con 0x7f078c15cea0 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- 172.21.15.146:0/238606503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c15cea0 msgr2=0x7f078c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 --2- 172.21.15.146:0/238606503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c15cea0 0x7f078c15d2e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f078c067ef0 tx=0x7f0784009780 comp rx=0 tx=0).stop 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- 172.21.15.146:0/238606503 shutdown_connections 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 --2- 172.21.15.146:0/238606503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c15cea0 0x7f078c15d2e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 --2- 172.21.15.146:0/238606503 >> v2:172.21.15.161:3300/0 conn(0x7f078c0d0010 0x7f078c0d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- 172.21.15.146:0/238606503 >> 172.21.15.146:0/238606503 conn(0x7f078c068250 msgr2=0x7f078c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- 172.21.15.146:0/238606503 shutdown_connections 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- 172.21.15.146:0/238606503 wait complete. 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 Processor -- start 2024-08-15T17:00:24.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.753+0000 7f078b59e700 1 -- start start 2024-08-15T17:00:24.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0d0010 0x7f078c1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 0x7f078c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c16dba0 con 0x7f078c159040 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c16cfd0 con 0x7f078c15cea0 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c16d270 con 0x7f078c0d0010 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37956/0 (socket says 172.21.15.146:37956) 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078ad9d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 0x7f078c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 -- 172.21.15.146:0/141593308 learned_addr learned my addr 172.21.15.146:0/141593308 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:24.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078a59c700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0d0010 0x7f078c1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 -- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0d0010 msgr2=0x7f078c1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0d0010 0x7f078c1449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 -- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 msgr2=0x7f078c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:24.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 0x7f078c15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:24.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 -- 172.21.15.146:0/141593308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009a20 con 0x7f078c159040 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078ad9d700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 0x7f078c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f0789d9b700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f078000e930 tx=0x7f07800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780013680 con 0x7f078c159040 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07800137e0 con 0x7f078c159040 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c16d550 con 0x7f078c159040 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780013b80 con 0x7f078c159040 2024-08-15T17:00:24.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.757+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c144ef0 con 0x7f078c159040 2024-08-15T17:00:24.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c16dd50 con 0x7f078c159040 2024-08-15T17:00:24.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f078000c3e0 con 0x7f078c159040 2024-08-15T17:00:24.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f077a7fc700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 0x7f076c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:24.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f078007c8a0 con 0x7f078c159040 2024-08-15T17:00:24.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f078a59c700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 0x7f076c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:24.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.761+0000 7f078a59c700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 0x7f076c062d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f077c007fd0 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:24.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.765+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f078c16dd50 con 0x7f078c159040 2024-08-15T17:00:25.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:24.993+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f078c0643a0 con 0x7f078c159040 2024-08-15T17:00:25.012 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f077a7fc700 1 -- 172.21.15.146:0/141593308 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f078004d040 con 0x7f078c159040 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 msgr2=0x7f076c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 0x7f076c062d10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f077c007fd0 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 msgr2=0x7f078c15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f078000e930 tx=0x7f07800091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 shutdown_connections 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f076c060860 0x7f076c062d10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c15cea0 0x7f078c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f078c159040 0x7f078c15c1b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 --2- 172.21.15.146:0/141593308 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0d0010 0x7f078c1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 >> 172.21.15.146:0/141593308 conn(0x7f078c068250 msgr2=0x7f078c0b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 shutdown_connections 2024-08-15T17:00:25.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.001+0000 7f078b59e700 1 -- 172.21.15.146:0/141593308 wait complete. 2024-08-15T17:00:25.028 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-08-15T17:00:25.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.129+0000 7f8445454700 1 Processor -- start 2024-08-15T17:00:25.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 -- start start 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8440158f00 0x7f84400b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84400b0aa0 0x7f84400b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84400b4050 0x7f84400b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400b4480 con 0x7f8440158f00 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400b6020 con 0x7f84400b4050 2024-08-15T17:00:25.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f8445454700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400b1020 con 0x7f84400b0aa0 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8440158f00 0x7f84400b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8440158f00 0x7f84400b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37960/0 (socket says 172.21.15.146:37960) 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 -- 172.21.15.146:0/4000559088 learned_addr learned my addr 172.21.15.146:0/4000559088 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843f7fe700 1 --2- 172.21.15.146:0/4000559088 >> v2:172.21.15.161:3300/0 conn(0x7f84400b4050 0x7f84400b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843e7fc700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84400b0aa0 0x7f84400b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 -- 172.21.15.146:0/4000559088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84400b0aa0 msgr2=0x7f84400b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84400b0aa0 0x7f84400b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 -- 172.21.15.146:0/4000559088 >> v2:172.21.15.161:3300/0 conn(0x7f84400b4050 msgr2=0x7f84400b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 --2- 172.21.15.146:0/4000559088 >> v2:172.21.15.161:3300/0 conn(0x7f84400b4050 0x7f84400b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 -- 172.21.15.146:0/4000559088 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400b1300 con 0x7f8440158f00 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843effd700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8440158f00 0x7f84400b0560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84400b0f20 tx=0x7f8430009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2735caf95c6ac27e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843dffb700 1 -- 172.21.15.146:0/4000559088 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000b640 con 0x7f8440158f00 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843dffb700 1 -- 172.21.15.146:0/4000559088 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843000b7a0 con 0x7f8440158f00 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.133+0000 7f843dffb700 1 -- 172.21.15.146:0/4000559088 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000bb60 con 0x7f8440158f00 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/4000559088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8440158f00 msgr2=0x7f84400b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8440158f00 0x7f84400b0560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84400b0f20 tx=0x7f8430009250 comp rx=0 tx=0).stop 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/4000559088 shutdown_connections 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- 172.21.15.146:0/4000559088 >> v2:172.21.15.161:3300/0 conn(0x7f84400b4050 0x7f84400b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84400b0aa0 0x7f84400b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- 172.21.15.146:0/4000559088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8440158f00 0x7f84400b0560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/4000559088 >> 172.21.15.146:0/4000559088 conn(0x7f8440099510 msgr2=0x7f8440067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/4000559088 shutdown_connections 2024-08-15T17:00:25.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/4000559088 wait complete. 2024-08-15T17:00:25.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 Processor -- start 2024-08-15T17:00:25.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- start start 2024-08-15T17:00:25.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84400b0aa0 0x7f8440144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 0x7f8440068430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400ad2d0 con 0x7f84400b4050 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400ad5b0 con 0x7f84400b0aa0 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400ad890 con 0x7f8440158f00 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37970/0 (socket says 172.21.15.146:37970) 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 -- 172.21.15.146:0/3395696543 learned_addr learned my addr 172.21.15.146:0/3395696543 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843f7fe700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 0x7f8440068430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843effd700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84400b0aa0 0x7f8440144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 -- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 msgr2=0x7f8440068430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 0x7f8440068430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 -- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84400b0aa0 msgr2=0x7f8440144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84400b0aa0 0x7f8440144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 -- 172.21.15.146:0/3395696543 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84300094f0 con 0x7f84400b4050 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843f7fe700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 0x7f8440068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f843e7fc700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f843400e950 tx=0x7f84340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400c850 con 0x7f84400b4050 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400c9b0 con 0x7f84400b4050 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400adb70 con 0x7f84400b4050 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400cdb0 con 0x7f84400b4050 2024-08-15T17:00:25.153 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.137+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8440145530 con 0x7f84400b4050 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84400645c0 con 0x7f84400b4050 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8434013680 con 0x7f84400b4050 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f8426ffd700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 0x7f842c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f843407ca60 con 0x7f84400b4050 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f843effd700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 0x7f842c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.141+0000 7f843effd700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 0x7f842c062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84400b0f20 tx=0x7f8430017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.160 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.149+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f843404d200 con 0x7f84400b4050 2024-08-15T17:00:25.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.373+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f84400b0880 con 0x7f84400b4050 2024-08-15T17:00:25.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.381+0000 7f8426ffd700 1 -- 172.21.15.146:0/3395696543 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8434050820 con 0x7f84400b4050 2024-08-15T17:00:25.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 msgr2=0x7f842c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 0x7f842c062d10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84400b0f20 tx=0x7f8430017000 comp rx=0 tx=0).stop 2024-08-15T17:00:25.397 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 msgr2=0x7f8440145920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f843400e950 tx=0x7f84340091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 shutdown_connections 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f842c060860 0x7f842c062d10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8440158f00 0x7f8440068430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f84400b4050 0x7f8440145920 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 --2- 172.21.15.146:0/3395696543 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84400b0aa0 0x7f8440144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 >> 172.21.15.146:0/3395696543 conn(0x7f8440099510 msgr2=0x7f8440157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 shutdown_connections 2024-08-15T17:00:25.398 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.385+0000 7f8445454700 1 -- 172.21.15.146:0/3395696543 wait complete. 2024-08-15T17:00:25.410 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-08-15T17:00:25.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 Processor -- start 2024-08-15T17:00:25.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 -- start start 2024-08-15T17:00:25.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8190154ba0 0x7f8190157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81900b0dc0 0x7f81900afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81900ca2c0 con 0x7f81900b0120 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81900b15d0 con 0x7f8190154ba0 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81972d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8190154f70 con 0x7f81900b0dc0 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8190154ba0 0x7f8190157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8190154ba0 0x7f8190157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37972/0 (socket says 172.21.15.146:37972) 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 -- 172.21.15.146:0/3291367841 learned_addr learned my addr 172.21.15.146:0/3291367841 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f8195ace700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81900b0dc0 0x7f81900afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 -- 172.21.15.146:0/3291367841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81900b0dc0 msgr2=0x7f81900afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81900b0dc0 0x7f81900afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 -- 172.21.15.146:0/3291367841 >> v1:172.21.15.161:6789/0 conn(0x7f81900b0120 legacy=0x7f819015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.527 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 -- 172.21.15.146:0/3291367841 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190158410 con 0x7f8190154ba0 2024-08-15T17:00:25.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81952cd700 1 -- 172.21.15.146:0/3291367841 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929396343 0 0) 0x7f81900ca2c0 con 0x7f81900b0120 2024-08-15T17:00:25.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.513+0000 7f81962cf700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8190154ba0 0x7f8190157ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f20 tx=0x7f818c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0db94e87cafcde7 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81952cd700 1 -- 172.21.15.146:0/3291367841 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c00e070 con 0x7f8190154ba0 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81952cd700 1 -- 172.21.15.146:0/3291367841 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c00b9d0 con 0x7f8190154ba0 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81952cd700 1 -- 172.21.15.146:0/3291367841 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0092e0 con 0x7f8190154ba0 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 -- 172.21.15.146:0/3291367841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8190154ba0 msgr2=0x7f8190157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8190154ba0 0x7f8190157ed0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f818c009f20 tx=0x7f818c009840 comp rx=0 tx=0).stop 2024-08-15T17:00:25.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 -- 172.21.15.146:0/3291367841 shutdown_connections 2024-08-15T17:00:25.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81900b0dc0 0x7f81900afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 --2- 172.21.15.146:0/3291367841 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8190154ba0 0x7f8190157ed0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 -- 172.21.15.146:0/3291367841 >> 172.21.15.146:0/3291367841 conn(0x7f81900ca930 msgr2=0x7f81900cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 -- 172.21.15.146:0/3291367841 shutdown_connections 2024-08-15T17:00:25.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.517+0000 7f81972d1700 1 -- 172.21.15.146:0/3291367841 wait complete. 2024-08-15T17:00:25.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 Processor -- start 2024-08-15T17:00:25.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- start start 2024-08-15T17:00:25.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 0x7f8190166b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8190154ba0 0x7f81900c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819014af40 con 0x7f81900b0dc0 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81900c7cb0 con 0x7f81900b0120 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819014a330 con 0x7f8190154ba0 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37978/0 (socket says 172.21.15.146:37978) 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 -- 172.21.15.146:0/1394827896 learned_addr learned my addr 172.21.15.146:0/1394827896 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81962cf700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 0x7f8190166b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8196ad0700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8190154ba0 0x7f81900c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 -- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8190154ba0 msgr2=0x7f81900c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8190154ba0 0x7f81900c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 -- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 msgr2=0x7f8190166b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 0x7f8190166b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 -- 172.21.15.146:0/1394827896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009ae0 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81962cf700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 0x7f8190166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f8195ace700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f818000ec80 tx=0x7f81800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000c9d0 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000cb50 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180013470 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819014a610 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.521+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819014abd0 con 0x7f81900b0dc0 2024-08-15T17:00:25.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8190156b20 con 0x7f81900b0dc0 2024-08-15T17:00:25.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8180013860 con 0x7f81900b0dc0 2024-08-15T17:00:25.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81867fc700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 0x7f8174062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f818007cc20 con 0x7f81900b0dc0 2024-08-15T17:00:25.538 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81962cf700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 0x7f8174062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.525+0000 7f81962cf700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 0x7f8174062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f818c009880 tx=0x7f818c018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.533+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f818004d320 con 0x7f81900b0dc0 2024-08-15T17:00:25.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.753+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f81900ca6a0 con 0x7f81900b0dc0 2024-08-15T17:00:25.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.761+0000 7f81867fc700 1 -- 172.21.15.146:0/1394827896 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8180050940 con 0x7f81900b0dc0 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 msgr2=0x7f8174062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 0x7f8174062d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f818c009880 tx=0x7f818c018000 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 msgr2=0x7f819014b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f818000ec80 tx=0x7f81800091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 shutdown_connections 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8174060860 0x7f8174062d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8190154ba0 0x7f81900c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f81900b0dc0 0x7f819014b3d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 --2- 172.21.15.146:0/1394827896 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81900b0120 0x7f8190166b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 >> 172.21.15.146:0/1394827896 conn(0x7f81900ca930 msgr2=0x7f81900c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 shutdown_connections 2024-08-15T17:00:25.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.765+0000 7f81972d1700 1 -- 172.21.15.146:0/1394827896 wait complete. 2024-08-15T17:00:25.791 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:25.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 Processor -- start 2024-08-15T17:00:25.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 -- start start 2024-08-15T17:00:25.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e54159040 0x7f0e540cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540cf190 con 0x7f0e5415d5b0 2024-08-15T17:00:25.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e540ce5e0 con 0x7f0e540d0010 2024-08-15T17:00:25.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5415d9f0 con 0x7f0e54159040 2024-08-15T17:00:25.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0e54159040 0x7f0e540cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e52d9d700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0e5415d5b0 0x7f0e5415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45332/0 (socket says 172.21.15.146:45332) 2024-08-15T17:00:25.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e52d9d700 1 -- 172.21.15.146:0/1260434821 learned_addr learned my addr 172.21.15.146:0/1260434821 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:25.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5259c700 1 -- 172.21.15.146:0/1260434821 >> v1:172.21.15.161:6789/0 conn(0x7f0e540d0010 legacy=0x7f0e5415cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:25.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5259c700 1 -- 172.21.15.146:0/1260434821 >> v1:172.21.15.146:6789/0 conn(0x7f0e5415d5b0 legacy=0x7f0e5415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5259c700 1 -- 172.21.15.146:0/1260434821 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540ce860 con 0x7f0e54159040 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5259c700 1 --2- 172.21.15.146:0/1260434821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e54159040 0x7f0e540cf620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e5415db30 tx=0x7f0e3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896a960943baf0b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5159a700 1 -- 172.21.15.146:0/1260434821 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00b670 con 0x7f0e54159040 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5159a700 1 -- 172.21.15.146:0/1260434821 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c00b7d0 con 0x7f0e54159040 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.881+0000 7f0e5159a700 1 -- 172.21.15.146:0/1260434821 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00bb70 con 0x7f0e54159040 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- 172.21.15.146:0/1260434821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e54159040 msgr2=0x7f0e540cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 --2- 172.21.15.146:0/1260434821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e54159040 0x7f0e540cf620 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e5415db30 tx=0x7f0e3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- 172.21.15.146:0/1260434821 shutdown_connections 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 --2- 172.21.15.146:0/1260434821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e54159040 0x7f0e540cf620 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- 172.21.15.146:0/1260434821 >> 172.21.15.146:0/1260434821 conn(0x7f0e54068250 msgr2=0x7f0e54068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- 172.21.15.146:0/1260434821 shutdown_connections 2024-08-15T17:00:25.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- 172.21.15.146:0/1260434821 wait complete. 2024-08-15T17:00:25.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 Processor -- start 2024-08-15T17:00:25.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.885+0000 7f0e5359e700 1 -- start start 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e54159040 0x7f0e5416adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e5415d5b0 0x7f0e5414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e540b09f0 con 0x7f0e540d0010 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e540b1290 con 0x7f0e5415d5b0 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5416b300 con 0x7f0e54159040 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:37996/0 (socket says 172.21.15.146:37996) 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 -- 172.21.15.146:0/3781134744 learned_addr learned my addr 172.21.15.146:0/3781134744 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e51d9b700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e54159040 0x7f0e5416adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e52d9d700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e5415d5b0 0x7f0e5414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 -- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e54159040 msgr2=0x7f0e5416adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e54159040 0x7f0e5416adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 -- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e5415d5b0 msgr2=0x7f0e5414d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e5415d5b0 0x7f0e5414d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:25.902 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 -- 172.21.15.146:0/3781134744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009420 con 0x7f0e540d0010 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5259c700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e540aee10 tx=0x7f0e3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00bb50 con 0x7f0e540d0010 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c011420 con 0x7f0e540d0010 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5414da70 con 0x7f0e540d0010 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0117c0 con 0x7f0e540d0010 2024-08-15T17:00:25.903 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.889+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e540b0f20 con 0x7f0e540d0010 2024-08-15T17:00:25.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e540643a0 con 0x7f0e540d0010 2024-08-15T17:00:25.904 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0e3c011bb0 con 0x7f0e540d0010 2024-08-15T17:00:25.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e4a7fc700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 0x7f0e34062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:25.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0e3c07e040 con 0x7f0e540d0010 2024-08-15T17:00:25.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e51d9b700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 0x7f0e34062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:25.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.893+0000 7f0e51d9b700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 0x7f0e34062d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e540cd290 tx=0x7f0e44009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:25.912 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:25.901+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0e3c04e7e0 con 0x7f0e540d0010 2024-08-15T17:00:26.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.065+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e540cfdf0 con 0x7f0e540d0010 2024-08-15T17:00:26.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e4a7fc700 1 -- 172.21.15.146:0/3781134744 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f0e3c011e60 con 0x7f0e540d0010 2024-08-15T17:00:26.080 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:26.080 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":28,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:23.156889+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:26.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 msgr2=0x7f0e34062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 0x7f0e34062d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0e540cd290 tx=0x7f0e44009060 comp rx=0 tx=0).stop 2024-08-15T17:00:26.082 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 msgr2=0x7f0e5414aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0e540aee10 tx=0x7f0e3c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 shutdown_connections 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0e34060860 0x7f0e34062d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e5415d5b0 0x7f0e5414d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e54159040 0x7f0e5416adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 --2- 172.21.15.146:0/3781134744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0e540d0010 0x7f0e5414aed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 >> 172.21.15.146:0/3781134744 conn(0x7f0e54068250 msgr2=0x7f0e54063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 shutdown_connections 2024-08-15T17:00:26.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.069+0000 7f0e5359e700 1 -- 172.21.15.146:0/3781134744 wait complete. 2024-08-15T17:00:26.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-15T17:00:26.095 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:00:26.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 Processor -- start 2024-08-15T17:00:26.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 -- start start 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413fa50 0x7f30040aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30040ad6c0 0x7f300406e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040b0810 con 0x7f300413f680 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30040afc70 con 0x7f30040ad6c0 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30091ca700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30040af060 con 0x7f300413fa50 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30040ad6c0 0x7f300406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f30040ad6c0 0x7f300406e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38006/0 (socket says 172.21.15.146:38006) 2024-08-15T17:00:26.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 -- 172.21.15.146:0/377915407 learned_addr learned my addr 172.21.15.146:0/377915407 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30037fe700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413fa50 0x7f30040aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 -- 172.21.15.146:0/377915407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413fa50 msgr2=0x7f30040aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413fa50 0x7f30040aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 -- 172.21.15.146:0/377915407 >> v1:172.21.15.161:6789/0 conn(0x7f300413f680 legacy=0x7f30040b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 -- 172.21.15.146:0/377915407 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040af340 con 0x7f30040ad6c0 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.197+0000 7f30089c9700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30040ad6c0 0x7f300406e010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0099b0 tx=0x7f2ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7011b718f5d2736d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f3002ffd700 1 -- 172.21.15.146:0/377915407 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00b860 con 0x7f30040ad6c0 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f3002ffd700 1 -- 172.21.15.146:0/377915407 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc00b9c0 con 0x7f30040ad6c0 2024-08-15T17:00:26.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f3002ffd700 1 -- 172.21.15.146:0/377915407 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00bd60 con 0x7f30040ad6c0 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- 172.21.15.146:0/377915407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30040ad6c0 msgr2=0x7f300406e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30040ad6c0 0x7f300406e010 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0099b0 tx=0x7f2ffc009250 comp rx=0 tx=0).stop 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- 172.21.15.146:0/377915407 shutdown_connections 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30040ad6c0 0x7f300406e010 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 --2- 172.21.15.146:0/377915407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413fa50 0x7f30040aea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- 172.21.15.146:0/377915407 >> 172.21.15.146:0/377915407 conn(0x7f30040678a0 msgr2=0x7f3004067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- 172.21.15.146:0/377915407 shutdown_connections 2024-08-15T17:00:26.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- 172.21.15.146:0/377915407 wait complete. 2024-08-15T17:00:26.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 Processor -- start 2024-08-15T17:00:26.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 -- start start 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.201+0000 7f30091ca700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30040ad6c0 0x7f3004158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413f680 0x7f300413b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3004134980 con 0x7f300413fa50 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3004133db0 con 0x7f30040ad6c0 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3004134050 con 0x7f300413f680 2024-08-15T17:00:26.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38022/0 (socket says 172.21.15.146:38022) 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 -- 172.21.15.146:0/1244391314 learned_addr learned my addr 172.21.15.146:0/1244391314 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f3003fff700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30040ad6c0 0x7f3004158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30037fe700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413f680 0x7f300413b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 -- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413f680 msgr2=0x7f300413b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413f680 0x7f300413b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 -- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30040ad6c0 msgr2=0x7f3004158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30040ad6c0 0x7f3004158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 -- 172.21.15.146:0/1244391314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0094f0 con 0x7f300413fa50 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30089c9700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3004068b10 tx=0x7f2ffc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00b860 con 0x7f300413fa50 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc01a630 con 0x7f300413fa50 2024-08-15T17:00:26.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3004134330 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc01a9d0 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3004134e10 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004134af0 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2ffc02d440 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2ffc07e360 con 0x7f300413fa50 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.205+0000 7f3003fff700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.209+0000 7f3003fff700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 0x7f2fe4062d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2fec003a80 tx=0x7f2fec007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.213+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2ffc04eae0 con 0x7f300413fa50 2024-08-15T17:00:26.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.377+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3004064330 con 0x7f300413fa50 2024-08-15T17:00:26.391 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f2ffbfff700 1 -- 172.21.15.146:0/1244391314 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ffc052100 con 0x7f300413fa50 2024-08-15T17:00:26.392 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:00:26.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 msgr2=0x7f2fe4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 0x7f2fe4062d10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2fec003a80 tx=0x7f2fec007ec0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 msgr2=0x7f300413bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3004068b10 tx=0x7f2ffc017000 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 shutdown_connections 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2fe4060860 0x7f2fe4062d10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f300413fa50 0x7f300413bcb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f300413f680 0x7f300413b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 --2- 172.21.15.146:0/1244391314 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30040ad6c0 0x7f3004158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 >> 172.21.15.146:0/1244391314 conn(0x7f30040678a0 msgr2=0x7f300406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 shutdown_connections 2024-08-15T17:00:26.395 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.381+0000 7f30091ca700 1 -- 172.21.15.146:0/1244391314 wait complete. 2024-08-15T17:00:26.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-15T17:00:26.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 Processor -- start 2024-08-15T17:00:26.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 -- start start 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd894142ae0 0x7fd8940cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940c9800 con 0x7fd89415c5e0 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8940c9ae0 con 0x7fd89415c9b0 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd899e06700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8940cc4c0 con 0x7fd894142ae0 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd8937fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd89415c5e0 0x7fd8940c8fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45346/0 (socket says 172.21.15.146:45346) 2024-08-15T17:00:26.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd8937fe700 1 -- 172.21.15.146:0/3693318700 learned_addr learned my addr 172.21.15.146:0/3693318700 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:26.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd893fff700 1 --2- 172.21.15.146:0/3693318700 >> v2:172.21.15.161:3300/0 conn(0x7fd894142ae0 0x7fd8940cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd8927fc700 1 -- 172.21.15.146:0/3693318700 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202418462 0 0) 0x7fd8940c9800 con 0x7fd89415c5e0 2024-08-15T17:00:26.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd8927fc700 1 -- 172.21.15.146:0/3693318700 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd874003570 con 0x7fd89415c5e0 2024-08-15T17:00:26.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd893fff700 1 -- 172.21.15.146:0/3693318700 >> v1:172.21.15.161:6789/0 conn(0x7fd89415c9b0 legacy=0x7fd8941423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd893fff700 1 -- 172.21.15.146:0/3693318700 >> v1:172.21.15.146:6789/0 conn(0x7fd89415c5e0 legacy=0x7fd8940c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.497+0000 7fd893fff700 1 -- 172.21.15.146:0/3693318700 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940c9dc0 con 0x7fd894142ae0 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd893fff700 1 --2- 172.21.15.146:0/3693318700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd894142ae0 0x7fd8940cbf80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e60 tx=0x7fd88c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8f8188e6c5f11e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd8927fc700 1 -- 172.21.15.146:0/3693318700 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c00e070 con 0x7fd894142ae0 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd8927fc700 1 -- 172.21.15.146:0/3693318700 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c00bb60 con 0x7fd894142ae0 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd8927fc700 1 -- 172.21.15.146:0/3693318700 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c012430 con 0x7fd894142ae0 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- 172.21.15.146:0/3693318700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd894142ae0 msgr2=0x7fd8940cbf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 --2- 172.21.15.146:0/3693318700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd894142ae0 0x7fd8940cbf80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009e60 tx=0x7fd88c009780 comp rx=0 tx=0).stop 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- 172.21.15.146:0/3693318700 shutdown_connections 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 --2- 172.21.15.146:0/3693318700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd894142ae0 0x7fd8940cbf80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- 172.21.15.146:0/3693318700 >> 172.21.15.146:0/3693318700 conn(0x7fd8940caad0 msgr2=0x7fd8940caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- 172.21.15.146:0/3693318700 shutdown_connections 2024-08-15T17:00:26.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- 172.21.15.146:0/3693318700 wait complete. 2024-08-15T17:00:26.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 Processor -- start 2024-08-15T17:00:26.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.501+0000 7fd899e06700 1 -- start start 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd894142ae0 0x7fd89415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd89415c5e0 0x7fd8940b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940c6b30 con 0x7fd89415c9b0 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940b0e60 con 0x7fd89415c5e0 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8940b19e0 con 0x7fd894142ae0 2024-08-15T17:00:26.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38030/0 (socket says 172.21.15.146:38030) 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 -- 172.21.15.146:0/2181038727 learned_addr learned my addr 172.21.15.146:0/2181038727 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd892ffd700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd89415c5e0 0x7fd8940b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd8937fe700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd894142ae0 0x7fd89415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 -- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd894142ae0 msgr2=0x7fd89415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd894142ae0 0x7fd89415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 -- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd89415c5e0 msgr2=0x7fd8940b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd89415c5e0 0x7fd8940b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 -- 172.21.15.146:0/2181038727 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c009a20 con 0x7fd89415c9b0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd893fff700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd88c013000 tx=0x7fd88c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c027470 con 0x7fd89415c9b0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c0275d0 con 0x7fd89415c9b0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8940b10b0 con 0x7fd89415c9b0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c027990 con 0x7fd89415c9b0 2024-08-15T17:00:26.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8940b1670 con 0x7fd89415c9b0 2024-08-15T17:00:26.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.505+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8940b1b50 con 0x7fd89415c9b0 2024-08-15T17:00:26.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.509+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd88c024070 con 0x7fd89415c9b0 2024-08-15T17:00:26.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.509+0000 7fd88b7fe700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 0x7fd878062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.509+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd88c07f170 con 0x7fd89415c9b0 2024-08-15T17:00:26.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.509+0000 7fd8937fe700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 0x7fd878062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.509+0000 7fd8937fe700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 0x7fd878062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd884007fd0 tx=0x7fd884009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.517+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd88c04f910 con 0x7fd89415c9b0 2024-08-15T17:00:26.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.677+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd8940645c0 con 0x7fd89415c9b0 2024-08-15T17:00:26.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.677+0000 7fd88b7fe700 1 -- 172.21.15.146:0/2181038727 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd88c052f30 con 0x7fd89415c9b0 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout:[ 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout: "poolnum": 1, 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout: "poolname": ".mgr" 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:26.689 INFO:teuthology.orchestra.run.smithi146.stdout:] 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 msgr2=0x7fd878062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 0x7fd878062d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd884007fd0 tx=0x7fd884009040 comp rx=0 tx=0).stop 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 msgr2=0x7fd8940b24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd88c013000 tx=0x7fd88c019000 comp rx=0 tx=0).stop 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 shutdown_connections 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd878060860 0x7fd878062d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd89415c9b0 0x7fd8940b24c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd89415c5e0 0x7fd8940b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 --2- 172.21.15.146:0/2181038727 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd894142ae0 0x7fd89415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 >> 172.21.15.146:0/2181038727 conn(0x7fd8940caad0 msgr2=0x7fd8940684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 shutdown_connections 2024-08-15T17:00:26.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.681+0000 7fd899e06700 1 -- 172.21.15.146:0/2181038727 wait complete. 2024-08-15T17:00:26.707 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-15T17:00:26.707 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-15T17:00:26.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 Processor -- start 2024-08-15T17:00:26.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 -- start start 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe4440b26d0 0x7fe4440b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 0x7fe4440c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440c5b40 con 0x7fe4440b26d0 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b2aa0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.809+0000 7fe443fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444159820 con 0x7fe4440c6990 2024-08-15T17:00:26.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe442ffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe4440b26d0 0x7fe4440b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4427fc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 0x7fe4440c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:42852/0 (socket says 172.21.15.146:42852) 2024-08-15T17:00:26.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 -- 172.21.15.146:0/765057160 learned_addr learned my addr 172.21.15.146:0/765057160 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 -- 172.21.15.146:0/765057160 >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 msgr2=0x7fe4440c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 --2- 172.21.15.146:0/765057160 >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 0x7fe4440c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 -- 172.21.15.146:0/765057160 >> v2:172.21.15.146:3300/0 conn(0x7fe4440b26d0 msgr2=0x7fe4440b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 --2- 172.21.15.146:0/765057160 >> v2:172.21.15.146:3300/0 conn(0x7fe4440b26d0 0x7fe4440b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 -- 172.21.15.146:0/765057160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440c4f30 con 0x7fe4440c6990 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4427fc700 1 --2- 172.21.15.146:0/765057160 >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 0x7fe4440c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:26.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe4437fe700 1 --2- 172.21.15.146:0/765057160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009f20 tx=0x7fe43c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8baca2bfb75bf65 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe441ffb700 1 -- 172.21.15.146:0/765057160 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c00e070 con 0x7fe4440c6990 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe441ffb700 1 -- 172.21.15.146:0/765057160 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c00b930 con 0x7fe4440c6990 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe441ffb700 1 -- 172.21.15.146:0/765057160 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c0092e0 con 0x7fe4440c6990 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 -- 172.21.15.146:0/765057160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 msgr2=0x7fe4440cacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 --2- 172.21.15.146:0/765057160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009f20 tx=0x7fe43c009840 comp rx=0 tx=0).stop 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 -- 172.21.15.146:0/765057160 shutdown_connections 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 --2- 172.21.15.146:0/765057160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c6990 0x7fe4440cacd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 --2- 172.21.15.146:0/765057160 >> v2:172.21.15.161:3300/0 conn(0x7fe4440c5fd0 0x7fe4440c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 --2- 172.21.15.146:0/765057160 >> v2:172.21.15.146:3300/0 conn(0x7fe4440b26d0 0x7fe4440b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 -- 172.21.15.146:0/765057160 >> 172.21.15.146:0/765057160 conn(0x7fe444067590 msgr2=0x7fe4440d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 -- 172.21.15.146:0/765057160 shutdown_connections 2024-08-15T17:00:26.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.813+0000 7fe443fff700 1 -- 172.21.15.146:0/765057160 wait complete. 2024-08-15T17:00:26.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 Processor -- start 2024-08-15T17:00:26.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 -- start start 2024-08-15T17:00:26.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4440b26d0 0x7fe444168f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 0x7fe44416bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 0x7fe444153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440cbb60 con 0x7fe4440c6990 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe444152150 con 0x7fe4440b26d0 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440cbd40 con 0x7fe4440c5fd0 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 0x7fe444153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 0x7fe44416bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4437fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 0x7fe444153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38050/0 (socket says 172.21.15.146:38050) 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4437fe700 1 -- 172.21.15.146:0/1267401551 learned_addr learned my addr 172.21.15.146:0/1267401551 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:26.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe442ffd700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4440b26d0 0x7fe444168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 -- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4440b26d0 msgr2=0x7fe444168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4440b26d0 0x7fe444168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 -- 172.21.15.146:0/1267401551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 msgr2=0x7fe444153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 0x7fe444153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 -- 172.21.15.146:0/1267401551 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340093c0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe4427fc700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 0x7fe44416bbe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea50 tx=0x7fe434009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434005a00 con 0x7fe4440c5fd0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434005b60 con 0x7fe4440c5fd0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43400c5e0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.817+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c009ae0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.821+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4440cc5e0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.821+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4440644b0 con 0x7fe4440c5fd0 2024-08-15T17:00:26.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.821+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe43400c990 con 0x7fe4440c5fd0 2024-08-15T17:00:26.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.821+0000 7fe42affd700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 0x7fe430062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:26.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.821+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe43407cf90 con 0x7fe4440c5fd0 2024-08-15T17:00:26.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.825+0000 7fe442ffd700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 0x7fe430062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:26.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.825+0000 7fe442ffd700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 0x7fe430062d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe444154750 tx=0x7fe4380091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:26.845 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.833+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe43404d730 con 0x7fe4440c5fd0 2024-08-15T17:00:27.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:26.993+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe4440cc020 con 0x7fe4440c5fd0 2024-08-15T17:00:27.412 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.401+0000 7fe42affd700 1 -- 172.21.15.146:0/1267401551 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe4340200b0 con 0x7fe4440c5fd0 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 msgr2=0x7fe430062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 0x7fe430062d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe444154750 tx=0x7fe4380091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 msgr2=0x7fe44416bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 0x7fe44416bbe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe43400ea50 tx=0x7fe434009120 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 shutdown_connections 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe430060890 0x7fe430062d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4440c6990 0x7fe444153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4440c5fd0 0x7fe44416bbe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 --2- 172.21.15.146:0/1267401551 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4440b26d0 0x7fe444168f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 >> 172.21.15.146:0/1267401551 conn(0x7fe444067590 msgr2=0x7fe4440b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 shutdown_connections 2024-08-15T17:00:27.417 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.405+0000 7fe443fff700 1 -- 172.21.15.146:0/1267401551 wait complete. 2024-08-15T17:00:27.418 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' created 2024-08-15T17:00:27.429 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-15T17:00:27.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.533+0000 7f2b2cc74700 1 Processor -- start 2024-08-15T17:00:27.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.533+0000 7f2b2cc74700 1 -- start start 2024-08-15T17:00:27.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b280ac290 0x7f2b280ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.548 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b280ad480 con 0x7f2b281562a0 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b28069020 con 0x7f2b280ac290 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280ac870 con 0x7f2b280686a0 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b26d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b280ac290 0x7f2b280ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b26d9d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:42876/0 (socket says 172.21.15.146:42876) 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b26d9d700 1 -- 172.21.15.146:0/2855963087 learned_addr learned my addr 172.21.15.146:0/2855963087 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:27.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2659c700 1 --1- 172.21.15.146:0/2855963087 >> v1:172.21.15.161:6789/0 conn(0x7f2b281562a0 0x7f2b2806bf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:49298/0 (socket says 172.21.15.146:49298) 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 -- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 msgr2=0x7f2b28068ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 -- 172.21.15.146:0/2855963087 >> v1:172.21.15.161:6789/0 conn(0x7f2b281562a0 legacy=0x7f2b2806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 -- 172.21.15.146:0/2855963087 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280acb50 con 0x7f2b280ac290 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b26d9d700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b1dd9b700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ad910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098e0 tx=0x7f2b100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78c610c043d51d46 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b25d9b700 1 -- 172.21.15.146:0/2855963087 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011630 con 0x7f2b280ac290 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b25d9b700 1 -- 172.21.15.146:0/2855963087 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b10011790 con 0x7f2b280ac290 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b25d9b700 1 -- 172.21.15.146:0/2855963087 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011b30 con 0x7f2b280ac290 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 msgr2=0x7f2b280ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ad910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b100098e0 tx=0x7f2b100091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- 172.21.15.146:0/2855963087 shutdown_connections 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b28068ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/2855963087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ad910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- 172.21.15.146:0/2855963087 >> 172.21.15.146:0/2855963087 conn(0x7f2b2806db30 msgr2=0x7f2b2806df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- 172.21.15.146:0/2855963087 shutdown_connections 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.537+0000 7f2b2cc74700 1 -- 172.21.15.146:0/2855963087 wait complete. 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 Processor -- start 2024-08-15T17:00:27.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 -- start start 2024-08-15T17:00:27.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b281530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.553 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2815d910 con 0x7f2b281562a0 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28153930 con 0x7f2b280ac290 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2cc74700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2815cfe0 con 0x7f2b280686a0 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b26d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b281530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b1dd9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b26d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:38060/0 (socket says 172.21.15.146:38060) 2024-08-15T17:00:27.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b26d9d700 1 -- 172.21.15.146:0/1469841590 learned_addr learned my addr 172.21.15.146:0/1469841590 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 -- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 msgr2=0x7f2b280ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 -- 172.21.15.146:0/1469841590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 msgr2=0x7f2b28169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:27.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 -- 172.21.15.146:0/1469841590 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2000bcf0 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b1dd9b700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b26d9d700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b2659c700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b281530d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c5e0 tx=0x7f2b18007e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.541+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18005a70 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18005bd0 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b10009420 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180070d0 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2815d5a0 con 0x7f2b280686a0 2024-08-15T17:00:27.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b280645c0 con 0x7f2b280686a0 2024-08-15T17:00:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2b18007500 con 0x7f2b280686a0 2024-08-15T17:00:27.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.545+0000 7f2b1effd700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 0x7f2b08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:27.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.549+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2b1807bdb0 con 0x7f2b280686a0 2024-08-15T17:00:27.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.549+0000 7f2b1dd9b700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 0x7f2b08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:27.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.549+0000 7f2b1dd9b700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 0x7f2b08062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b100091e0 tx=0x7f2b10016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:27.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.557+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2b1804c3e0 con 0x7f2b280686a0 2024-08-15T17:00:27.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:27.721+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2b2815d120 con 0x7f2b280686a0 2024-08-15T17:00:28.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.417+0000 7f2b1effd700 1 -- 172.21.15.146:0/1469841590 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2b2815d120 con 0x7f2b280686a0 2024-08-15T17:00:28.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 msgr2=0x7f2b08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 0x7f2b08062d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2b100091e0 tx=0x7f2b10016000 comp rx=0 tx=0).stop 2024-08-15T17:00:28.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 msgr2=0x7f2b281530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b281530d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c5e0 tx=0x7f2b18007e30 comp rx=0 tx=0).stop 2024-08-15T17:00:28.439 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 shutdown_connections 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2b08060860 0x7f2b08062d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b281562a0 0x7f2b28169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b280ac290 0x7f2b280ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 --2- 172.21.15.146:0/1469841590 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b280686a0 0x7f2b281530d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 >> 172.21.15.146:0/1469841590 conn(0x7f2b2806db30 msgr2=0x7f2b280c8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.425+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 shutdown_connections 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.429+0000 7f2b2cc74700 1 -- 172.21.15.146:0/1469841590 wait complete. 2024-08-15T17:00:28.440 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' created 2024-08-15T17:00:28.450 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-15T17:00:28.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.541+0000 7f37948de700 1 Processor -- start 2024-08-15T17:00:28.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.541+0000 7f37948de700 1 -- start start 2024-08-15T17:00:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f379013fa50 0x7f37900ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37900afc70 con 0x7f37900b0100 2024-08-15T17:00:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900adc00 con 0x7f379013fa50 2024-08-15T17:00:28.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900af060 con 0x7f379013f680 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378ffff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f37900b0100 0x7f379006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45398/0 (socket says 172.21.15.146:45398) 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378ffff700 1 -- 172.21.15.146:0/2918885327 learned_addr learned my addr 172.21.15.146:0/2918885327 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 --2- 172.21.15.146:0/2918885327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378effd700 1 --2- 172.21.15.146:0/2918885327 >> v2:172.21.15.161:3300/0 conn(0x7f379013fa50 0x7f37900ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684128754 0 0) 0x7f37900afc70 con 0x7f37900b0100 2024-08-15T17:00:28.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3778003570 con 0x7f37900b0100 2024-08-15T17:00:28.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 -- 172.21.15.146:0/2918885327 >> v2:172.21.15.161:3300/0 conn(0x7f379013fa50 msgr2=0x7f37900ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:28.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 --2- 172.21.15.146:0/2918885327 >> v2:172.21.15.161:3300/0 conn(0x7f379013fa50 0x7f37900ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397011539 0 0) 0x7f3778003570 con 0x7f37900b0100 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 -- 172.21.15.146:0/2918885327 >> v1:172.21.15.146:6789/0 conn(0x7f37900b0100 legacy=0x7f379006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 -- 172.21.15.146:0/2918885327 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900af340 con 0x7f379013f680 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378f7fe700 1 --2- 172.21.15.146:0/2918885327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900aea80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e60 tx=0x7f3788009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5febe559f08ec301 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378800e070 con 0x7f379013f680 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378800b7a0 con 0x7f379013f680 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f378e7fc700 1 -- 172.21.15.146:0/2918885327 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780001470 con 0x7f379013f680 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- 172.21.15.146:0/2918885327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 msgr2=0x7f37900aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 --2- 172.21.15.146:0/2918885327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900aea80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e60 tx=0x7f3788009780 comp rx=0 tx=0).stop 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- 172.21.15.146:0/2918885327 shutdown_connections 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 --2- 172.21.15.146:0/2918885327 >> v2:172.21.15.161:3300/0 conn(0x7f379013fa50 0x7f37900ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 --2- 172.21.15.146:0/2918885327 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900aea80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.559 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- 172.21.15.146:0/2918885327 >> 172.21.15.146:0/2918885327 conn(0x7f37900678a0 msgr2=0x7f3790067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:28.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- 172.21.15.146:0/2918885327 shutdown_connections 2024-08-15T17:00:28.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.545+0000 7f37948de700 1 -- 172.21.15.146:0/2918885327 wait complete. 2024-08-15T17:00:28.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 Processor -- start 2024-08-15T17:00:28.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 -- start start 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 0x7f3790157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f379013fa50 0x7f37900abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900a75b0 con 0x7f37900b0100 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900a7890 con 0x7f379013fa50 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37948de700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900a7b70 con 0x7f379013f680 2024-08-15T17:00:28.562 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 0x7f3790157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378f7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 0x7f3790157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50818/0 (socket says 172.21.15.146:50818) 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378f7fe700 1 -- 172.21.15.146:0/3632197920 learned_addr learned my addr 172.21.15.146:0/3632197920 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378ffff700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f379013fa50 0x7f37900abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 -- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f379013fa50 msgr2=0x7f37900abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f379013fa50 0x7f37900abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 -- 172.21.15.146:0/3632197920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 msgr2=0x7f3790157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 0x7f3790157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:28.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 -- 172.21.15.146:0/3632197920 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788009a20 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f378effd700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900ab6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f378400e8d0 tx=0x7f37840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.549+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784009450 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3784013680 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37900a7e50 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784013a40 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37900a8410 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3790064330 con 0x7f379013f680 2024-08-15T17:00:28.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f37840123d0 con 0x7f379013f680 2024-08-15T17:00:28.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37777fe700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 0x7f377c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:28.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f378407cb50 con 0x7f379013f680 2024-08-15T17:00:28.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.553+0000 7f378f7fe700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 0x7f377c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:28.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.557+0000 7f378f7fe700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 0x7f377c062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37880097c0 tx=0x7f3788018000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:28.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.565+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3784050a90 con 0x7f379013f680 2024-08-15T17:00:28.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:28.765+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f379006ed70 con 0x7f379013f680 2024-08-15T17:00:29.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.413+0000 7f37777fe700 1 -- 172.21.15.146:0/3632197920 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3784050620 con 0x7f379013f680 2024-08-15T17:00:29.443 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.429+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 msgr2=0x7f377c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 0x7f377c062d10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f37880097c0 tx=0x7f3788018000 comp rx=0 tx=0).stop 2024-08-15T17:00:29.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 msgr2=0x7f37900ab6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900ab6e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f378400e8d0 tx=0x7f37840091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 shutdown_connections 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f377c060860 0x7f377c062d10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f379013fa50 0x7f37900abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f379013f680 0x7f37900ab6e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 --2- 172.21.15.146:0/3632197920 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f37900b0100 0x7f3790157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 >> 172.21.15.146:0/3632197920 conn(0x7f37900678a0 msgr2=0x7f379013d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:29.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 shutdown_connections 2024-08-15T17:00:29.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.433+0000 7f37948de700 1 -- 172.21.15.146:0/3632197920 wait complete. 2024-08-15T17:00:29.448 INFO:teuthology.orchestra.run.smithi146.stderr:new fs with metadata pool 4 and data pool 5 2024-08-15T17:00:29.461 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:29.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.549+0000 7fcf1a36c700 1 Processor -- start 2024-08-15T17:00:29.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.549+0000 7fcf1a36c700 1 -- start start 2024-08-15T17:00:29.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1a36c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141553e0 0x7fcf140af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.565 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1a36c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf140686a0 0x7fcf14068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1a36c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf14155c00 con 0x7fcf141557b0 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1a36c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf14069060 con 0x7fcf140686a0 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1a36c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14155e30 con 0x7fcf141553e0 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf140686a0 0x7fcf14068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcf140686a0 0x7fcf14068b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50830/0 (socket says 172.21.15.146:50830) 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 -- 172.21.15.146:0/143989836 learned_addr learned my addr 172.21.15.146:0/143989836 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:29.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf1936a700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141553e0 0x7fcf140af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 -- 172.21.15.146:0/143989836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141553e0 msgr2=0x7fcf140af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141553e0 0x7fcf140af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 -- 172.21.15.146:0/143989836 >> v1:172.21.15.161:6789/0 conn(0x7fcf141557b0 legacy=0x7fcf140b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 -- 172.21.15.146:0/143989836 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14156110 con 0x7fcf140686a0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf19b6b700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf14068b20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00d1d0 tx=0x7fcf0c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a3c0952ce293fa server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf0bfff700 1 -- 172.21.15.146:0/143989836 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c010070 con 0x7fcf140686a0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf0bfff700 1 -- 172.21.15.146:0/143989836 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00dbf0 con 0x7fcf140686a0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.553+0000 7fcf0bfff700 1 -- 172.21.15.146:0/143989836 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c016400 con 0x7fcf140686a0 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- 172.21.15.146:0/143989836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 msgr2=0x7fcf14068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf14068b20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00d1d0 tx=0x7fcf0c00ba50 comp rx=0 tx=0).stop 2024-08-15T17:00:29.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- 172.21.15.146:0/143989836 shutdown_connections 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf14068b20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/143989836 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141553e0 0x7fcf140af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- 172.21.15.146:0/143989836 >> 172.21.15.146:0/143989836 conn(0x7fcf1406d800 msgr2=0x7fcf1406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- 172.21.15.146:0/143989836 shutdown_connections 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- 172.21.15.146:0/143989836 wait complete. 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 Processor -- start 2024-08-15T17:00:29.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- start start 2024-08-15T17:00:29.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf141553e0 0x7fcf141654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141557b0 0x7fcf1414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14165a00 con 0x7fcf140686a0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1414f620 con 0x7fcf141553e0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1a36c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141533a0 con 0x7fcf141557b0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1936a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1936a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50832/0 (socket says 172.21.15.146:50832) 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.557+0000 7fcf1936a700 1 -- 172.21.15.146:0/3533820466 learned_addr learned my addr 172.21.15.146:0/3533820466 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf18b69700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf141553e0 0x7fcf141654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf19b6b700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141557b0 0x7fcf1414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 -- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141557b0 msgr2=0x7fcf1414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141557b0 0x7fcf1414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 -- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf141553e0 msgr2=0x7fcf141654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf141553e0 0x7fcf141654c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 -- 172.21.15.146:0/3533820466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c00bcf0 con 0x7fcf140686a0 2024-08-15T17:00:29.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1936a700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000ec80 tx=0x7fcf100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000c850 con 0x7fcf140686a0 2024-08-15T17:00:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1000c9b0 con 0x7fcf140686a0 2024-08-15T17:00:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf14152790 con 0x7fcf140686a0 2024-08-15T17:00:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000cd50 con 0x7fcf140686a0 2024-08-15T17:00:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf14152d50 con 0x7fcf140686a0 2024-08-15T17:00:29.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf140a9500 con 0x7fcf140686a0 2024-08-15T17:00:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcf10013680 con 0x7fcf140686a0 2024-08-15T17:00:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 0x7fcef8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.561+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf1007cdd0 con 0x7fcf140686a0 2024-08-15T17:00:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.565+0000 7fcf18b69700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 0x7fcef8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.565+0000 7fcf18b69700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 0x7fcef8062d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf141625e0 tx=0x7fcf04007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.569+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcf1004d220 con 0x7fcf140686a0 2024-08-15T17:00:29.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.737+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf14063920 con 0x7fcf140686a0 2024-08-15T17:00:29.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.737+0000 7fcf097fa700 1 -- 172.21.15.146:0/3533820466 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fcf10050840 con 0x7fcf140686a0 2024-08-15T17:00:29.750 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:29.750 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":31,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:28.769874+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:29.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 msgr2=0x7fcef8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 0x7fcef8062d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf141625e0 tx=0x7fcf04007ec0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 msgr2=0x7fcf141521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000ec80 tx=0x7fcf100091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 shutdown_connections 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcef8060890 0x7fcef8062d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf141557b0 0x7fcf1414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf141553e0 0x7fcf141654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 --2- 172.21.15.146:0/3533820466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf140686a0 0x7fcf141521b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 >> 172.21.15.146:0/3533820466 conn(0x7fcf1406d800 msgr2=0x7fcf141507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 shutdown_connections 2024-08-15T17:00:29.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.741+0000 7fcf1a36c700 1 -- 172.21.15.146:0/3533820466 wait complete. 2024-08-15T17:00:29.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:29.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.861+0000 7f8e04f3a700 1 Processor -- start 2024-08-15T17:00:29.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.861+0000 7f8e04f3a700 1 -- start start 2024-08-15T17:00:29.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8e04f3a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e00155fb0 0x7f8e00156380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8e04f3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e000687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8e04f3a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0006a3e0 con 0x7f8e00068ce0 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8e04f3a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e000697d0 con 0x7f8e00155fb0 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8e04f3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00069ab0 con 0x7f8e000b0050 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e00155fb0 0x7f8e00156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8e00155fb0 0x7f8e00156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50836/0 (socket says 172.21.15.146:50836) 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 -- 172.21.15.146:0/2455661769 learned_addr learned my addr 172.21.15.146:0/2455661769 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e000687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 -- 172.21.15.146:0/2455661769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 msgr2=0x7f8e000687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e000687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 -- 172.21.15.146:0/2455661769 >> v1:172.21.15.146:6789/0 conn(0x7f8e00068ce0 legacy=0x7f8e000c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 -- 172.21.15.146:0/2455661769 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00069d90 con 0x7f8e00155fb0 2024-08-15T17:00:29.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e000687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:29.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dff7fe700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00155fb0 0x7f8e00156380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8df00099b0 tx=0x7f8df0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9854cec06e52acf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dfeffd700 1 -- 172.21.15.146:0/2455661769 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000b860 con 0x7f8e00155fb0 2024-08-15T17:00:29.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dfeffd700 1 -- 172.21.15.146:0/2455661769 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000b9c0 con 0x7f8e00155fb0 2024-08-15T17:00:29.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.865+0000 7f8dfeffd700 1 -- 172.21.15.146:0/2455661769 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df000bd60 con 0x7f8e00155fb0 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- 172.21.15.146:0/2455661769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00155fb0 msgr2=0x7f8e00156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00155fb0 0x7f8e00156380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8df00099b0 tx=0x7f8df0009250 comp rx=0 tx=0).stop 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- 172.21.15.146:0/2455661769 shutdown_connections 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e000687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/2455661769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00155fb0 0x7f8e00156380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- 172.21.15.146:0/2455661769 >> 172.21.15.146:0/2455661769 conn(0x7f8e000ccfc0 msgr2=0x7f8e000c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- 172.21.15.146:0/2455661769 shutdown_connections 2024-08-15T17:00:29.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- 172.21.15.146:0/2455661769 wait complete. 2024-08-15T17:00:29.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 Processor -- start 2024-08-15T17:00:29.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.869+0000 7f8e04f3a700 1 -- start start 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e001587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e00155fb0 0x7f8e00152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00147a70 con 0x7f8e00068ce0 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e00158350 con 0x7f8e00155fb0 2024-08-15T17:00:29.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0014bf40 con 0x7f8e000b0050 2024-08-15T17:00:29.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8dff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8dff7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50848/0 (socket says 172.21.15.146:50848) 2024-08-15T17:00:29.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8dff7fe700 1 -- 172.21.15.146:0/4033045165 learned_addr learned my addr 172.21.15.146:0/4033045165 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:29.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e001587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8dfffff700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e00155fb0 0x7f8e00152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 -- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e00155fb0 msgr2=0x7f8e00152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e00155fb0 0x7f8e00152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 -- 172.21.15.146:0/4033045165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 msgr2=0x7f8e00147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 -- 172.21.15.146:0/4033045165 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00094f0 con 0x7f8e000b0050 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8dff7fe700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:29.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df6ffd700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e001587e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c880 tx=0x7f8de8008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8011600 con 0x7f8e000b0050 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de8011760 con 0x7f8e000b0050 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e00157d00 con 0x7f8e000b0050 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8011b00 con 0x7f8e000b0050 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e00158d20 con 0x7f8e000b0050 2024-08-15T17:00:29.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.873+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e001584e0 con 0x7f8e000b0050 2024-08-15T17:00:29.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.877+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8de8018360 con 0x7f8e000b0050 2024-08-15T17:00:29.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.877+0000 7f8df7fff700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 0x7f8de0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:29.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.877+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8de807cb10 con 0x7f8e000b0050 2024-08-15T17:00:29.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.877+0000 7f8dff7fe700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 0x7f8de0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:29.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.877+0000 7f8dff7fe700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 0x7f8de0062d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8df0006d20 tx=0x7f8df0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:29.900 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:29.889+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8de804cf60 con 0x7f8e000b0050 2024-08-15T17:00:30.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.053+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e000638f0 con 0x7f8e000b0050 2024-08-15T17:00:30.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8df7fff700 1 -- 172.21.15.146:0/4033045165 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f8de8050580 con 0x7f8e000b0050 2024-08-15T17:00:30.067 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:30.067 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":31,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:28.769874+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 msgr2=0x7f8de0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 0x7f8de0062d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8df0006d20 tx=0x7f8df0017000 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 msgr2=0x7f8e001587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e001587e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c880 tx=0x7f8de8008040 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 shutdown_connections 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8de0060860 0x7f8de0062d10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e00155fb0 0x7f8e00152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e000b0050 0x7f8e001587e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 --2- 172.21.15.146:0/4033045165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e00068ce0 0x7f8e00147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 >> 172.21.15.146:0/4033045165 conn(0x7f8e000ccfc0 msgr2=0x7f8e0014df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 shutdown_connections 2024-08-15T17:00:30.070 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.057+0000 7f8e04f3a700 1 -- 172.21.15.146:0/4033045165 wait complete. 2024-08-15T17:00:30.082 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-15T17:00:30.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.173+0000 7fcaa31c3700 1 Processor -- start 2024-08-15T17:00:30.185 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.173+0000 7fcaa31c3700 1 -- start start 2024-08-15T17:00:30.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fca9c1552a0 0x7fca9c0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.187 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0ae490 con 0x7fca9c068be0 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c0ae920 con 0x7fca9c1552a0 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0af040 con 0x7fca9c0b2370 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa29c2700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fca9c068be0 0x7fca9c159190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45420/0 (socket says 172.21.15.146:45420) 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa29c2700 1 -- 172.21.15.146:0/1317328690 learned_addr learned my addr 172.21.15.146:0/1317328690 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa19c0700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.188 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/1317328690 >> v2:172.21.15.161:3300/0 conn(0x7fca9c1552a0 0x7fca9c0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa11bf700 1 -- 172.21.15.146:0/1317328690 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571283150 0 0) 0x7fca9c0ae490 con 0x7fca9c068be0 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa11bf700 1 -- 172.21.15.146:0/1317328690 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca88003570 con 0x7fca9c068be0 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 -- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 msgr2=0x7fca9c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c0686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 -- 172.21.15.146:0/1317328690 >> v1:172.21.15.146:6789/0 conn(0x7fca9c068be0 legacy=0x7fca9c159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 -- 172.21.15.146:0/1317328690 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c0ad880 con 0x7fca9c1552a0 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c0b1e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca9800a990 tx=0x7fca980080e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109a022a1ba070d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:30.189 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa11bf700 1 -- 172.21.15.146:0/1317328690 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800b530 con 0x7fca9c1552a0 2024-08-15T17:00:30.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa11bf700 1 -- 172.21.15.146:0/1317328690 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800b690 con 0x7fca9c1552a0 2024-08-15T17:00:30.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa11bf700 1 -- 172.21.15.146:0/1317328690 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ba10 con 0x7fca9c1552a0 2024-08-15T17:00:30.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 msgr2=0x7fca9c0b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.190 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c0b1e30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca9800a990 tx=0x7fca980080e0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- 172.21.15.146:0/1317328690 shutdown_connections 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c0686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/1317328690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c0b1e30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- 172.21.15.146:0/1317328690 >> 172.21.15.146:0/1317328690 conn(0x7fca9c06e350 msgr2=0x7fca9c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- 172.21.15.146:0/1317328690 shutdown_connections 2024-08-15T17:00:30.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.177+0000 7fcaa31c3700 1 -- 172.21.15.146:0/1317328690 wait complete. 2024-08-15T17:00:30.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 Processor -- start 2024-08-15T17:00:30.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- start start 2024-08-15T17:00:30.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c16c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c16ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c16ae10 con 0x7fca9c068be0 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c13e4d0 con 0x7fca9c1552a0 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c14f800 con 0x7fca9c0b2370 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50864/0 (socket says 172.21.15.146:50864) 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 -- 172.21.15.146:0/3721590030 learned_addr learned my addr 172.21.15.146:0/3721590030 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa19c0700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c16c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa29c2700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c16ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 -- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 msgr2=0x7fca9c16c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c16c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 -- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 msgr2=0x7fca9c16ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c16ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 -- 172.21.15.146:0/3721590030 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9800a4d0 con 0x7fca9c068be0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa19c0700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c16c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa21c1700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca9800af80 tx=0x7fca98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9801f070 con 0x7fca9c068be0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9800be50 con 0x7fca9c068be0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c16a4e0 con 0x7fca9c068be0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800ba10 con 0x7fca9c068be0 2024-08-15T17:00:30.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.181+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c16aaa0 con 0x7fca9c068be0 2024-08-15T17:00:30.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c0638f0 con 0x7fca9c068be0 2024-08-15T17:00:30.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fca98019450 con 0x7fca9c068be0 2024-08-15T17:00:30.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fca927fc700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 0x7fca80062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:30.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca9807e330 con 0x7fca9c068be0 2024-08-15T17:00:30.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fcaa19c0700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 0x7fca80062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:30.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.185+0000 7fcaa19c0700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 0x7fca80062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00b240 tx=0x7fca8c00b430 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:30.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.189+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fca9804e7e0 con 0x7fca9c068be0 2024-08-15T17:00:30.405 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:30.393+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fca9c0ad2f0 con 0x7fca9c068be0 2024-08-15T17:00:31.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.265+0000 7fca927fc700 1 -- 172.21.15.146:0/3721590030 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7fca98051e00 con 0x7fca9c068be0 2024-08-15T17:00:31.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 msgr2=0x7fca80062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 0x7fca80062d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00b240 tx=0x7fca8c00b430 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 msgr2=0x7fca9c13df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca9800af80 tx=0x7fca98019000 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 shutdown_connections 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fca80060860 0x7fca80062d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fca9c1552a0 0x7fca9c16ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fca9c0b2370 0x7fca9c16c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 --2- 172.21.15.146:0/3721590030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca9c068be0 0x7fca9c13df90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 >> 172.21.15.146:0/3721590030 conn(0x7fca9c06e350 msgr2=0x7fca9c153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 shutdown_connections 2024-08-15T17:00:31.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.269+0000 7fcaa31c3700 1 -- 172.21.15.146:0/3721590030 wait complete. 2024-08-15T17:00:31.296 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-15T17:00:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 Processor -- start 2024-08-15T17:00:31.419 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- start start 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc06f350 0x7f3bbc159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bbc0b0ad0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc15a940 con 0x7f3bbc1538e0 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0b01a0 con 0x7f3bbc06f350 2024-08-15T17:00:31.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50876/0 (socket says 172.21.15.146:50876) 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 -- 172.21.15.146:0/2353157773 learned_addr learned my addr 172.21.15.146:0/2353157773 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbbfff700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc06f350 0x7f3bbc159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 -- 172.21.15.146:0/2353157773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc06f350 msgr2=0x7f3bbc159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc06f350 0x7f3bbc159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 -- 172.21.15.146:0/2353157773 >> v1:172.21.15.161:6789/0 conn(0x7f3bbc153cb0 legacy=0x7f3bbc06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 -- 172.21.15.146:0/2353157773 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc0b0480 con 0x7f3bbc1538e0 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bbb7fe700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0099b0 tx=0x7f3bac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717deeacbf8b41ab server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bba7fc700 1 -- 172.21.15.146:0/2353157773 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00b860 con 0x7f3bbc1538e0 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bba7fc700 1 -- 172.21.15.146:0/2353157773 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac00b9c0 con 0x7f3bbc1538e0 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bba7fc700 1 -- 172.21.15.146:0/2353157773 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00bd60 con 0x7f3bbc1538e0 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- 172.21.15.146:0/2353157773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc1538e0 msgr2=0x7f3bbc0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0099b0 tx=0x7f3bac009250 comp rx=0 tx=0).stop 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- 172.21.15.146:0/2353157773 shutdown_connections 2024-08-15T17:00:31.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc06f350 0x7f3bbc159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 --2- 172.21.15.146:0/2353157773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc1538e0 0x7f3bbc0b0f60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- 172.21.15.146:0/2353157773 >> 172.21.15.146:0/2353157773 conn(0x7f3bbc066e00 msgr2=0x7f3bbc067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- 172.21.15.146:0/2353157773 shutdown_connections 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.409+0000 7f3bc209e700 1 -- 172.21.15.146:0/2353157773 wait complete. 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 Processor -- start 2024-08-15T17:00:31.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 -- start start 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 0x7f3bbc16a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc1538e0 0x7f3bbc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 0x7f3bbc0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c84e0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c9300 con 0x7f3bbc06f350 2024-08-15T17:00:31.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bc209e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0c86f0 con 0x7f3bbc1538e0 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 0x7f3bbc16a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 0x7f3bbc0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbaffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc1538e0 0x7f3bbc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 0x7f3bbc16a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:46782/0 (socket says 172.21.15.146:46782) 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbb7fe700 1 -- 172.21.15.146:0/3473019270 learned_addr learned my addr 172.21.15.146:0/3473019270 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbbfff700 1 -- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc1538e0 msgr2=0x7f3bbc0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbbfff700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc1538e0 0x7f3bbc0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbbfff700 1 -- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 msgr2=0x7f3bbc16a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.413+0000 7f3bbbfff700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 0x7f3bbc16a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bbbfff700 1 -- 172.21.15.146:0/3473019270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0094f0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bbbfff700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 0x7f3bbc0c9dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4007ea0 tx=0x7f3bb4009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4018070 con 0x7f3bbc153cb0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb400eac0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0c89d0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb400ee60 con 0x7f3bbc153cb0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc0c8f90 con 0x7f3bbc153cb0 2024-08-15T17:00:31.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc0645c0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3bb4003020 con 0x7f3bbc153cb0 2024-08-15T17:00:31.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 0x7f3ba8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:31.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3bb407ccb0 con 0x7f3bbc153cb0 2024-08-15T17:00:31.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.417+0000 7f3bbb7fe700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 0x7f3ba8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:31.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.421+0000 7f3bbb7fe700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 0x7f3ba8062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009fd0 tx=0x7f3bac017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:31.440 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.429+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3bb404d100 con 0x7f3bbc153cb0 2024-08-15T17:00:31.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:31.633+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3bbc1595e0 con 0x7f3bbc153cb0 2024-08-15T17:00:32.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.449+0000 7f3ba37fe700 1 -- 172.21.15.146:0/3473019270 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3bb4050720 con 0x7f3bbc153cb0 2024-08-15T17:00:32.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 msgr2=0x7f3ba8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 0x7f3ba8062d10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009fd0 tx=0x7f3bac017000 comp rx=0 tx=0).stop 2024-08-15T17:00:32.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 msgr2=0x7f3bbc0c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 0x7f3bbc0c9dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4007ea0 tx=0x7f3bb4009850 comp rx=0 tx=0).stop 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 shutdown_connections 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3ba8060860 0x7f3ba8062d10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bbc153cb0 0x7f3bbc0c9dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3bbc1538e0 0x7f3bbc0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 --2- 172.21.15.146:0/3473019270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3bbc06f350 0x7f3bbc16a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 >> 172.21.15.146:0/3473019270 conn(0x7f3bbc066e00 msgr2=0x7f3bbc1544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 shutdown_connections 2024-08-15T17:00:32.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.453+0000 7f3bc209e700 1 -- 172.21.15.146:0/3473019270 wait complete. 2024-08-15T17:00:32.480 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:32.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 Processor -- start 2024-08-15T17:00:32.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 -- start start 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 0x7f7000141e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7000141a00 con 0x7f70000b0d20 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000140b10 con 0x7f70000b1630 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005a13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7000140df0 con 0x7f700006e620 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7004a11700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f70000b0d20 0x7f700006df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45428/0 (socket says 172.21.15.146:45428) 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7004a11700 1 -- 172.21.15.146:0/2603723780 learned_addr learned my addr 172.21.15.146:0/2603723780 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005212700 1 --2- 172.21.15.146:0/2603723780 >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 0x7f7000141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 --2- 172.21.15.146:0/2603723780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6fff7fe700 1 -- 172.21.15.146:0/2603723780 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456086101 0 0) 0x7f7000141a00 con 0x7f70000b0d20 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6fff7fe700 1 -- 172.21.15.146:0/2603723780 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe0003570 con 0x7f70000b0d20 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 -- 172.21.15.146:0/2603723780 >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 msgr2=0x7f7000141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 --2- 172.21.15.146:0/2603723780 >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 0x7f7000141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 -- 172.21.15.146:0/2603723780 >> v1:172.21.15.146:6789/0 conn(0x7f70000b0d20 legacy=0x7f700006df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 -- 172.21.15.146:0/2603723780 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70001410d0 con 0x7f700006e620 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f7005212700 1 --2- 172.21.15.146:0/2603723780 >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 0x7f7000141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.581+0000 7f6ffffff700 1 --2- 172.21.15.146:0/2603723780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000b10f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009e60 tx=0x7f6ff8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a67d4c623f64e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6fff7fe700 1 -- 172.21.15.146:0/2603723780 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800e070 con 0x7f700006e620 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6fff7fe700 1 -- 172.21.15.146:0/2603723780 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff800bab0 con 0x7f700006e620 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6fff7fe700 1 -- 172.21.15.146:0/2603723780 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800be70 con 0x7f700006e620 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- 172.21.15.146:0/2603723780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 msgr2=0x7f70000b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- 172.21.15.146:0/2603723780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000b10f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009e60 tx=0x7f6ff8009780 comp rx=0 tx=0).stop 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- 172.21.15.146:0/2603723780 shutdown_connections 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- 172.21.15.146:0/2603723780 >> v2:172.21.15.146:3300/0 conn(0x7f70000b1630 0x7f7000141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- 172.21.15.146:0/2603723780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000b10f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- 172.21.15.146:0/2603723780 >> 172.21.15.146:0/2603723780 conn(0x7f70000678d0 msgr2=0x7f7000067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- 172.21.15.146:0/2603723780 shutdown_connections 2024-08-15T17:00:32.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- 172.21.15.146:0/2603723780 wait complete. 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 Processor -- start 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- start start 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 0x7f70001462b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000171fd0 con 0x7f70000b0d20 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000a71f0 con 0x7f70000b1630 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005a13700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70000a74d0 con 0x7f700006e620 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50914/0 (socket says 172.21.15.146:50914) 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f6ffffff700 1 -- 172.21.15.146:0/343893481 learned_addr learned my addr 172.21.15.146:0/343893481 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:32.600 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7004a11700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.585+0000 7f7005212700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 0x7f70001462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 -- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 msgr2=0x7f70000adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 -- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 msgr2=0x7f70001462b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 0x7f70001462b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 -- 172.21.15.146:0/343893481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8009420 con 0x7f70000b0d20 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f7005212700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 0x7f70001462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffffff700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7000140f30 tx=0x7f6ff8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8012e30 con 0x7f70000b0d20 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8027470 con 0x7f70000b0d20 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff8009a20 con 0x7f70000b0d20 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8027830 con 0x7f70000b0d20 2024-08-15T17:00:32.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70000a7a90 con 0x7f70000b0d20 2024-08-15T17:00:32.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.589+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70000a7ce0 con 0x7f70000b0d20 2024-08-15T17:00:32.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.593+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6ff8024070 con 0x7f70000b0d20 2024-08-15T17:00:32.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.593+0000 7f6ffcff9700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 0x7f6fe4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.593+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ff807f040 con 0x7f70000b0d20 2024-08-15T17:00:32.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.593+0000 7f7004a11700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 0x7f6fe4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.593+0000 7f7004a11700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 0x7f6fe4062d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0007fd0 tx=0x7f6ff0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.597+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6ff804f490 con 0x7f70000b0d20 2024-08-15T17:00:32.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.793+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70000643a0 con 0x7f70000b0d20 2024-08-15T17:00:32.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.793+0000 7f6ffcff9700 1 -- 172.21.15.146:0/343893481 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6ff8052ab0 con 0x7f70000b0d20 2024-08-15T17:00:32.806 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:32.807 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 msgr2=0x7f6fe4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 0x7f6fe4062d40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0007fd0 tx=0x7f6ff0009040 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 msgr2=0x7f7000145d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7000140f30 tx=0x7f6ff8019000 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 shutdown_connections 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6fe4060890 0x7f6fe4062d40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70000b1630 0x7f70001462b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70000b0d20 0x7f7000145d70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 --2- 172.21.15.146:0/343893481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f700006e620 0x7f70000adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 >> 172.21.15.146:0/343893481 conn(0x7f70000678d0 msgr2=0x7f700006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:32.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 shutdown_connections 2024-08-15T17:00:32.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.797+0000 7f7005a13700 1 -- 172.21.15.146:0/343893481 wait complete. 2024-08-15T17:00:32.810 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 14 2024-08-15T17:00:32.820 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:00:32.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 Processor -- start 2024-08-15T17:00:32.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 -- start start 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0aa806aff0 0x7f0aa81556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 0x7f0aa813e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa8156940 con 0x7f0aa813e790 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa813ec10 con 0x7f0aa806aff0 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aace20700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8155d30 con 0x7f0aa80b1660 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aa77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0aa806aff0 0x7f0aa81556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aa7fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f0aa813e790 0x7f0aa8158d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:43630/0 (socket says 172.21.15.146:43630) 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aa7fff700 1 -- 172.21.15.146:0/3705577413 learned_addr learned my addr 172.21.15.146:0/3705577413 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aa77fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0aa806aff0 0x7f0aa81556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50916/0 (socket says 172.21.15.146:50916) 2024-08-15T17:00:32.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.917+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 0x7f0aa813e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa67fc700 1 -- 172.21.15.146:0/3705577413 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604860106 0 0) 0x7f0aa8156940 con 0x7f0aa813e790 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa77fe700 1 -- 172.21.15.146:0/3705577413 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 msgr2=0x7f0aa813e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa77fe700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 0x7f0aa813e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa77fe700 1 -- 172.21.15.146:0/3705577413 >> v1:172.21.15.161:6789/0 conn(0x7f0aa813e790 legacy=0x7f0aa8158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa77fe700 1 -- 172.21.15.146:0/3705577413 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8156010 con 0x7f0aa806aff0 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 0x7f0aa813e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa77fe700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa806aff0 0x7f0aa81556f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098e0 tx=0x7f0a980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49efe9ad242547 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa67fc700 1 -- 172.21.15.146:0/3705577413 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800b640 con 0x7f0aa806aff0 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa67fc700 1 -- 172.21.15.146:0/3705577413 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9800b7a0 con 0x7f0aa806aff0 2024-08-15T17:00:32.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aa67fc700 1 -- 172.21.15.146:0/3705577413 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800bb40 con 0x7f0aa806aff0 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 -- 172.21.15.146:0/3705577413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa806aff0 msgr2=0x7f0aa81556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa806aff0 0x7f0aa81556f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0a980098e0 tx=0x7f0a980091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 -- 172.21.15.146:0/3705577413 shutdown_connections 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa80b1660 0x7f0aa813e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 --2- 172.21.15.146:0/3705577413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa806aff0 0x7f0aa81556f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 -- 172.21.15.146:0/3705577413 >> 172.21.15.146:0/3705577413 conn(0x7f0aa80678d0 msgr2=0x7f0aa8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 -- 172.21.15.146:0/3705577413 shutdown_connections 2024-08-15T17:00:32.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 -- 172.21.15.146:0/3705577413 wait complete. 2024-08-15T17:00:32.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.921+0000 7f0aace20700 1 Processor -- start 2024-08-15T17:00:32.935 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 -- start start 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa806aff0 0x7f0aa8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 0x7f0aa814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8154620 con 0x7f0aa813e790 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8153a10 con 0x7f0aa80b1660 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8153cf0 con 0x7f0aa806aff0 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50928/0 (socket says 172.21.15.146:50928) 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa7fff700 1 -- 172.21.15.146:0/2150144053 learned_addr learned my addr 172.21.15.146:0/2150144053 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:32.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 0x7f0aa814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa77fe700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa806aff0 0x7f0aa8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 -- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa806aff0 msgr2=0x7f0aa8154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa806aff0 0x7f0aa8154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 -- 172.21.15.146:0/2150144053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 msgr2=0x7f0aa814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 -- 172.21.15.146:0/2150144053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a90009420 con 0x7f0aa80b1660 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa7fff700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aa6ffd700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 0x7f0aa814e410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000ec80 tx=0x7f0a900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000c850 con 0x7f0aa80b1660 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9000c9b0 con 0x7f0aa80b1660 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000cd70 con 0x7f0aa80b1660 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.925+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98009420 con 0x7f0aa80b1660 2024-08-15T17:00:32.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.929+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa81542b0 con 0x7f0aa80b1660 2024-08-15T17:00:32.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.929+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa81547d0 con 0x7f0aa80b1660 2024-08-15T17:00:32.942 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.929+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0a90013680 con 0x7f0aa80b1660 2024-08-15T17:00:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.929+0000 7f0a9f7fe700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 0x7f0a88062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.933+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a9007cea0 con 0x7f0aa80b1660 2024-08-15T17:00:32.943 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.933+0000 7f0aa77fe700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 0x7f0a88062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:32.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.933+0000 7f0aa77fe700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 0x7f0a88062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a980093f0 tx=0x7f0a98017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:32.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:32.945+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f0aa81547d0 con 0x7f0aa80b1660 2024-08-15T17:00:33.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.105+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0aa81547d0 con 0x7f0aa80b1660 2024-08-15T17:00:33.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.109+0000 7f0a9f7fe700 1 -- 172.21.15.146:0/2150144053 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f0aa81547d0 con 0x7f0aa80b1660 2024-08-15T17:00:33.121 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:33.121 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":32,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:30.403137+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:00:33.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 msgr2=0x7f0a88062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 0x7f0a88062d10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0a980093f0 tx=0x7f0a98017000 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 msgr2=0x7f0aa814e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 0x7f0aa814e410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000ec80 tx=0x7f0a900091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 shutdown_connections 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0a88060860 0x7f0a88062d10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0aa813e790 0x7f0aa814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0aa80b1660 0x7f0aa814e410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 --2- 172.21.15.146:0/2150144053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0aa806aff0 0x7f0aa8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 >> 172.21.15.146:0/2150144053 conn(0x7f0aa80678d0 msgr2=0x7f0aa8137320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 shutdown_connections 2024-08-15T17:00:33.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.113+0000 7f0aace20700 1 -- 172.21.15.146:0/2150144053 wait complete. 2024-08-15T17:00:33.141 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-15T17:00:33.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:00:33.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 Processor -- start 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 -- start start 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c156940 con 0x7f3e5c13e250 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5c155d30 con 0x7f3e5c06e140 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e61f6b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c156010 con 0x7f3e5c06e510 2024-08-15T17:00:33.246 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e6176a700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f3e5c13e250 0x7f3e5c13e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45436/0 (socket says 172.21.15.146:45436) 2024-08-15T17:00:33.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e6176a700 1 -- 172.21.15.146:0/2195960533 learned_addr learned my addr 172.21.15.146:0/2195960533 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e5bfff700 1 --2- 172.21.15.146:0/2195960533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.233+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788934077 0 0) 0x7f3e5c156940 con 0x7f3e5c13e250 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e3c003570 con 0x7f3e5c13e250 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449352609 0 0) 0x7f3e5c155d30 con 0x7f3e5c06e140 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5bfff700 1 -- 172.21.15.146:0/2195960533 >> v1:172.21.15.161:6789/0 conn(0x7f3e5c06e140 legacy=0x7f3e5c0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5bfff700 1 -- 172.21.15.146:0/2195960533 >> v1:172.21.15.146:6789/0 conn(0x7f3e5c13e250 legacy=0x7f3e5c13e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5bfff700 1 -- 172.21.15.146:0/2195960533 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c1562f0 con 0x7f3e5c06e510 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5bfff700 1 --2- 172.21.15.146:0/2195960533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c1556f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009910 tx=0x7f3e440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8382ec4a82c4bf7a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400b680 con 0x7f3e5c06e510 2024-08-15T17:00:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b7e0 con 0x7f3e5c06e510 2024-08-15T17:00:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e5b7fe700 1 -- 172.21.15.146:0/2195960533 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400bb60 con 0x7f3e5c06e510 2024-08-15T17:00:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- 172.21.15.146:0/2195960533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 msgr2=0x7f3e5c1556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/2195960533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c1556f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3e44009910 tx=0x7f3e440091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- 172.21.15.146:0/2195960533 shutdown_connections 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/2195960533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c1556f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- 172.21.15.146:0/2195960533 >> 172.21.15.146:0/2195960533 conn(0x7f3e5c0678d0 msgr2=0x7f3e5c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- 172.21.15.146:0/2195960533 shutdown_connections 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- 172.21.15.146:0/2195960533 wait complete. 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 Processor -- start 2024-08-15T17:00:33.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.237+0000 7f3e61f6b700 1 -- start start 2024-08-15T17:00:33.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 0x7f3e5c153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c0ade10 con 0x7f3e5c13e250 2024-08-15T17:00:33.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c1670a0 con 0x7f3e5c06e140 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c167340 con 0x7f3e5c06e510 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e5bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50934/0 (socket says 172.21.15.146:50934) 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e5bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:50596/0 (socket says 172.21.15.146:50596) 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 -- 172.21.15.146:0/1617878430 learned_addr learned my addr 172.21.15.146:0/1617878430 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:33.252 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e60f69700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 0x7f3e5c153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 -- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 msgr2=0x7f3e5c0a7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 -- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 msgr2=0x7f3e5c153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 0x7f3e5c153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 -- 172.21.15.146:0/1617878430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009420 con 0x7f3e5c13e250 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e5bfff700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e60f69700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 0x7f3e5c153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e6176a700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400cbc0 tx=0x7f3e54001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400a7e0 con 0x7f3e5c13e250 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5400a940 con 0x7f3e5c13e250 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c167620 con 0x7f3e5c13e250 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5400ace0 con 0x7f3e5c13e250 2024-08-15T17:00:33.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.241+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c167e20 con 0x7f3e5c13e250 2024-08-15T17:00:33.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f3e54007300 con 0x7f3e5c13e250 2024-08-15T17:00:33.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c0643a0 con 0x7f3e5c13e250 2024-08-15T17:00:33.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e58ff9700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 0x7f3e40062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3e5407c870 con 0x7f3e5c13e250 2024-08-15T17:00:33.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e60f69700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 0x7f3e40062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.245+0000 7f3e60f69700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 0x7f3e40062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c007fd0 tx=0x7f3e4c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.253+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f3e5404ccc0 con 0x7f3e5c13e250 2024-08-15T17:00:33.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.417+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e5c06f5f0 con 0x7f3e40060860 2024-08-15T17:00:33.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.433+0000 7f3e58ff9700 1 -- 172.21.15.146:0/1617878430 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264526 (secure 0 0 0) 0x7f3e5c06f5f0 con 0x7f3e40060860 2024-08-15T17:00:33.450 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:33.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 msgr2=0x7f3e40062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 0x7f3e40062d10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c007fd0 tx=0x7f3e4c009040 comp rx=0 tx=0).stop 2024-08-15T17:00:33.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 msgr2=0x7f3e5c0a7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3e5400cbc0 tx=0x7f3e54001980 comp rx=0 tx=0).stop 2024-08-15T17:00:33.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 shutdown_connections 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f3e40060860 0x7f3e40062d10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3e5c13e250 0x7f3e5c0a7680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e5c06e510 0x7f3e5c0a7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 --2- 172.21.15.146:0/1617878430 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e5c06e140 0x7f3e5c153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 >> 172.21.15.146:0/1617878430 conn(0x7f3e5c0678d0 msgr2=0x7f3e5c063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 shutdown_connections 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.441+0000 7f3e61f6b700 1 -- 172.21.15.146:0/1617878430 wait complete. 2024-08-15T17:00:33.453 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:00:33.465 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-08-15T17:00:31.660461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386968,"kb_used_data":3932,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752587752,"statfs":{"total":773094113280,"available":770649858048,"internally_reserved":0,"allocated":4026368,"data_stored":2241515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.270007"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429898+0000","last_change":"2024-08-15T17:00:29.429898+0000","last_active":"2024-08-15T17:00:29.429898+0000","last_peered":"2024-08-15T17:00:29.429898+0000","last_clean":"2024-08-15T17:00:29.429898+0000","last_became_active":"2024-08-15T17:00:29.429650+0000","last_became_peered":"2024-08-15T17:00:29.429650+0000","last_unstale":"2024-08-15T17:00:29.429898+0000","last_undegraded":"2024-08-15T17:00:29.429898+0000","last_fullsized":"2024-08-15T17:00:29.429898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:50:24.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432521+0000","last_change":"2024-08-15T17:00:29.432521+0000","last_active":"2024-08-15T17:00:29.432521+0000","last_peered":"2024-08-15T17:00:29.432521+0000","last_clean":"2024-08-15T17:00:29.432521+0000","last_became_active":"2024-08-15T17:00:29.431901+0000","last_became_peered":"2024-08-15T17:00:29.431901+0000","last_unstale":"2024-08-15T17:00:29.432521+0000","last_undegraded":"2024-08-15T17:00:29.432521+0000","last_fullsized":"2024-08-15T17:00:29.432521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:48.751217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429949+0000","last_change":"2024-08-15T17:00:29.427098+0000","last_active":"2024-08-15T17:00:29.429949+0000","last_peered":"2024-08-15T17:00:29.429949+0000","last_clean":"2024-08-15T17:00:29.429949+0000","last_became_active":"2024-08-15T17:00:29.425645+0000","last_became_peered":"2024-08-15T17:00:29.425645+0000","last_unstale":"2024-08-15T17:00:29.429949+0000","last_undegraded":"2024-08-15T17:00:29.429949+0000","last_fullsized":"2024-08-15T17:00:29.429949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:30:40.296434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430965+0000","last_change":"2024-08-15T17:00:29.430965+0000","last_active":"2024-08-15T17:00:29.430965+0000","last_peered":"2024-08-15T17:00:29.430965+0000","last_clean":"2024-08-15T17:00:29.430965+0000","last_became_active":"2024-08-15T17:00:29.430705+0000","last_became_peered":"2024-08-15T17:00:29.430705+0000","last_unstale":"2024-08-15T17:00:29.430965+0000","last_undegraded":"2024-08-15T17:00:29.430965+0000","last_fullsized":"2024-08-15T17:00:29.430965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:46:21.695015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432193+0000","last_change":"2024-08-15T17:00:29.430770+0000","last_active":"2024-08-15T17:00:29.432193+0000","last_peered":"2024-08-15T17:00:29.432193+0000","last_clean":"2024-08-15T17:00:29.432193+0000","last_became_active":"2024-08-15T17:00:29.430216+0000","last_became_peered":"2024-08-15T17:00:29.430216+0000","last_unstale":"2024-08-15T17:00:29.432193+0000","last_undegraded":"2024-08-15T17:00:29.432193+0000","last_fullsized":"2024-08-15T17:00:29.432193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:10:09.167257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.428748+0000","last_change":"2024-08-15T17:00:29.428748+0000","last_active":"2024-08-15T17:00:29.428748+0000","last_peered":"2024-08-15T17:00:29.428748+0000","last_clean":"2024-08-15T17:00:29.428748+0000","last_became_active":"2024-08-15T17:00:29.428401+0000","last_became_peered":"2024-08-15T17:00:29.428401+0000","last_unstale":"2024-08-15T17:00:29.428748+0000","last_undegraded":"2024-08-15T17:00:29.428748+0000","last_fullsized":"2024-08-15T17:00:29.428748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:07.974975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425759+0000","last_change":"2024-08-15T17:00:29.425759+0000","last_active":"2024-08-15T17:00:29.425759+0000","last_peered":"2024-08-15T17:00:29.425759+0000","last_clean":"2024-08-15T17:00:29.425759+0000","last_became_active":"2024-08-15T17:00:29.425171+0000","last_became_peered":"2024-08-15T17:00:29.425171+0000","last_unstale":"2024-08-15T17:00:29.425759+0000","last_undegraded":"2024-08-15T17:00:29.425759+0000","last_fullsized":"2024-08-15T17:00:29.425759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:46:34.342266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423962+0000","last_change":"2024-08-15T17:00:29.423962+0000","last_active":"2024-08-15T17:00:29.423962+0000","last_peered":"2024-08-15T17:00:29.423962+0000","last_clean":"2024-08-15T17:00:29.423962+0000","last_became_active":"2024-08-15T17:00:29.422912+0000","last_became_peered":"2024-08-15T17:00:29.422912+0000","last_unstale":"2024-08-15T17:00:29.423962+0000","last_undegraded":"2024-08-15T17:00:29.423962+0000","last_fullsized":"2024-08-15T17:00:29.423962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:39.997135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430950+0000","last_change":"2024-08-15T17:00:29.430013+0000","last_active":"2024-08-15T17:00:29.430950+0000","last_peered":"2024-08-15T17:00:29.430950+0000","last_clean":"2024-08-15T17:00:29.430950+0000","last_became_active":"2024-08-15T17:00:29.429695+0000","last_became_peered":"2024-08-15T17:00:29.429695+0000","last_unstale":"2024-08-15T17:00:29.430950+0000","last_undegraded":"2024-08-15T17:00:29.430950+0000","last_fullsized":"2024-08-15T17:00:29.430950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:16:50.162772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.433699+0000","last_change":"2024-08-15T17:00:29.433699+0000","last_active":"2024-08-15T17:00:29.433699+0000","last_peered":"2024-08-15T17:00:29.433699+0000","last_clean":"2024-08-15T17:00:29.433699+0000","last_became_active":"2024-08-15T17:00:29.433380+0000","last_became_peered":"2024-08-15T17:00:29.433380+0000","last_unstale":"2024-08-15T17:00:29.433699+0000","last_undegraded":"2024-08-15T17:00:29.433699+0000","last_fullsized":"2024-08-15T17:00:29.433699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:27.365495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425374+0000","last_change":"2024-08-15T17:00:29.425374+0000","last_active":"2024-08-15T17:00:29.425374+0000","last_peered":"2024-08-15T17:00:29.425374+0000","last_clean":"2024-08-15T17:00:29.425374+0000","last_became_active":"2024-08-15T17:00:29.425071+0000","last_became_peered":"2024-08-15T17:00:29.425071+0000","last_unstale":"2024-08-15T17:00:29.425374+0000","last_undegraded":"2024-08-15T17:00:29.425374+0000","last_fullsized":"2024-08-15T17:00:29.425374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:38:05.927974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.436856+0000","last_change":"2024-08-15T17:00:29.433620+0000","last_active":"2024-08-15T17:00:29.436856+0000","last_peered":"2024-08-15T17:00:29.436856+0000","last_clean":"2024-08-15T17:00:29.436856+0000","last_became_active":"2024-08-15T17:00:29.432640+0000","last_became_peered":"2024-08-15T17:00:29.432640+0000","last_unstale":"2024-08-15T17:00:29.436856+0000","last_undegraded":"2024-08-15T17:00:29.436856+0000","last_fullsized":"2024-08-15T17:00:29.436856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:58:38.549577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426860+0000","last_change":"2024-08-15T17:00:29.424724+0000","last_active":"2024-08-15T17:00:29.426860+0000","last_peered":"2024-08-15T17:00:29.426860+0000","last_clean":"2024-08-15T17:00:29.426860+0000","last_became_active":"2024-08-15T17:00:29.424290+0000","last_became_peered":"2024-08-15T17:00:29.424290+0000","last_unstale":"2024-08-15T17:00:29.426860+0000","last_undegraded":"2024-08-15T17:00:29.426860+0000","last_fullsized":"2024-08-15T17:00:29.426860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:15:24.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425845+0000","last_change":"2024-08-15T17:00:29.425845+0000","last_active":"2024-08-15T17:00:29.425845+0000","last_peered":"2024-08-15T17:00:29.425845+0000","last_clean":"2024-08-15T17:00:29.425845+0000","last_became_active":"2024-08-15T17:00:29.425540+0000","last_became_peered":"2024-08-15T17:00:29.425540+0000","last_unstale":"2024-08-15T17:00:29.425845+0000","last_undegraded":"2024-08-15T17:00:29.425845+0000","last_fullsized":"2024-08-15T17:00:29.425845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:39.997135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429880+0000","last_change":"2024-08-15T17:00:29.427429+0000","last_active":"2024-08-15T17:00:29.429880+0000","last_peered":"2024-08-15T17:00:29.429880+0000","last_clean":"2024-08-15T17:00:29.429880+0000","last_became_active":"2024-08-15T17:00:29.427145+0000","last_became_peered":"2024-08-15T17:00:29.427145+0000","last_unstale":"2024-08-15T17:00:29.429880+0000","last_undegraded":"2024-08-15T17:00:29.429880+0000","last_fullsized":"2024-08-15T17:00:29.429880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:49.776176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431597+0000","last_change":"2024-08-15T17:00:29.431597+0000","last_active":"2024-08-15T17:00:29.431597+0000","last_peered":"2024-08-15T17:00:29.431597+0000","last_clean":"2024-08-15T17:00:29.431597+0000","last_became_active":"2024-08-15T17:00:29.431337+0000","last_became_peered":"2024-08-15T17:00:29.431337+0000","last_unstale":"2024-08-15T17:00:29.431597+0000","last_undegraded":"2024-08-15T17:00:29.431597+0000","last_fullsized":"2024-08-15T17:00:29.431597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:48.679569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426560+0000","last_change":"2024-08-15T17:00:29.426560+0000","last_active":"2024-08-15T17:00:29.426560+0000","last_peered":"2024-08-15T17:00:29.426560+0000","last_clean":"2024-08-15T17:00:29.426560+0000","last_became_active":"2024-08-15T17:00:29.426302+0000","last_became_peered":"2024-08-15T17:00:29.426302+0000","last_unstale":"2024-08-15T17:00:29.426560+0000","last_undegraded":"2024-08-15T17:00:29.426560+0000","last_fullsized":"2024-08-15T17:00:29.426560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:27.365495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430128+0000","last_change":"2024-08-15T17:00:29.430128+0000","last_active":"2024-08-15T17:00:29.430128+0000","last_peered":"2024-08-15T17:00:29.430128+0000","last_clean":"2024-08-15T17:00:29.430128+0000","last_became_active":"2024-08-15T17:00:29.429768+0000","last_became_peered":"2024-08-15T17:00:29.429768+0000","last_unstale":"2024-08-15T17:00:29.430128+0000","last_undegraded":"2024-08-15T17:00:29.430128+0000","last_fullsized":"2024-08-15T17:00:29.430128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:58.535553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432242+0000","last_change":"2024-08-15T17:00:29.432242+0000","last_active":"2024-08-15T17:00:29.432242+0000","last_peered":"2024-08-15T17:00:29.432242+0000","last_clean":"2024-08-15T17:00:29.432242+0000","last_became_active":"2024-08-15T17:00:29.431943+0000","last_became_peered":"2024-08-15T17:00:29.431943+0000","last_unstale":"2024-08-15T17:00:29.432242+0000","last_undegraded":"2024-08-15T17:00:29.432242+0000","last_fullsized":"2024-08-15T17:00:29.432242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:01.828223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423057+0000","last_change":"2024-08-15T17:00:29.421670+0000","last_active":"2024-08-15T17:00:29.423057+0000","last_peered":"2024-08-15T17:00:29.423057+0000","last_clean":"2024-08-15T17:00:29.423057+0000","last_became_active":"2024-08-15T17:00:29.421280+0000","last_became_peered":"2024-08-15T17:00:29.421280+0000","last_unstale":"2024-08-15T17:00:29.423057+0000","last_undegraded":"2024-08-15T17:00:29.423057+0000","last_fullsized":"2024-08-15T17:00:29.423057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:22:19.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430648+0000","last_change":"2024-08-15T17:00:29.427739+0000","last_active":"2024-08-15T17:00:29.430648+0000","last_peered":"2024-08-15T17:00:29.430648+0000","last_clean":"2024-08-15T17:00:29.430648+0000","last_became_active":"2024-08-15T17:00:29.427431+0000","last_became_peered":"2024-08-15T17:00:29.427431+0000","last_unstale":"2024-08-15T17:00:29.430648+0000","last_undegraded":"2024-08-15T17:00:29.430648+0000","last_fullsized":"2024-08-15T17:00:29.430648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:07.391680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437274+0000","last_change":"2024-08-15T17:00:29.432905+0000","last_active":"2024-08-15T17:00:29.437274+0000","last_peered":"2024-08-15T17:00:29.437274+0000","last_clean":"2024-08-15T17:00:29.437274+0000","last_became_active":"2024-08-15T17:00:29.432623+0000","last_became_peered":"2024-08-15T17:00:29.432623+0000","last_unstale":"2024-08-15T17:00:29.437274+0000","last_undegraded":"2024-08-15T17:00:29.437274+0000","last_fullsized":"2024-08-15T17:00:29.437274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:20:21.248696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422064+0000","last_change":"2024-08-15T17:00:29.422064+0000","last_active":"2024-08-15T17:00:29.422064+0000","last_peered":"2024-08-15T17:00:29.422064+0000","last_clean":"2024-08-15T17:00:29.422064+0000","last_became_active":"2024-08-15T17:00:29.421809+0000","last_became_peered":"2024-08-15T17:00:29.421809+0000","last_unstale":"2024-08-15T17:00:29.422064+0000","last_undegraded":"2024-08-15T17:00:29.422064+0000","last_fullsized":"2024-08-15T17:00:29.422064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:28:54.704212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426569+0000","last_change":"2024-08-15T17:00:29.421506+0000","last_active":"2024-08-15T17:00:29.426569+0000","last_peered":"2024-08-15T17:00:29.426569+0000","last_clean":"2024-08-15T17:00:29.426569+0000","last_became_active":"2024-08-15T17:00:29.421155+0000","last_became_peered":"2024-08-15T17:00:29.421155+0000","last_unstale":"2024-08-15T17:00:29.426569+0000","last_undegraded":"2024-08-15T17:00:29.426569+0000","last_fullsized":"2024-08-15T17:00:29.426569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:22:13.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.424387+0000","last_change":"2024-08-15T17:00:29.424387+0000","last_active":"2024-08-15T17:00:29.424387+0000","last_peered":"2024-08-15T17:00:29.424387+0000","last_clean":"2024-08-15T17:00:29.424387+0000","last_became_active":"2024-08-15T17:00:29.424096+0000","last_became_peered":"2024-08-15T17:00:29.424096+0000","last_unstale":"2024-08-15T17:00:29.424387+0000","last_undegraded":"2024-08-15T17:00:29.424387+0000","last_fullsized":"2024-08-15T17:00:29.424387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:36:10.787160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432782+0000","last_change":"2024-08-15T17:00:29.432782+0000","last_active":"2024-08-15T17:00:29.432782+0000","last_peered":"2024-08-15T17:00:29.432782+0000","last_clean":"2024-08-15T17:00:29.432782+0000","last_became_active":"2024-08-15T17:00:29.432265+0000","last_became_peered":"2024-08-15T17:00:29.432265+0000","last_unstale":"2024-08-15T17:00:29.432782+0000","last_undegraded":"2024-08-15T17:00:29.432782+0000","last_fullsized":"2024-08-15T17:00:29.432782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425813+0000","last_change":"2024-08-15T17:00:29.425813+0000","last_active":"2024-08-15T17:00:29.425813+0000","last_peered":"2024-08-15T17:00:29.425813+0000","last_clean":"2024-08-15T17:00:29.425813+0000","last_became_active":"2024-08-15T17:00:29.425448+0000","last_became_peered":"2024-08-15T17:00:29.425448+0000","last_unstale":"2024-08-15T17:00:29.425813+0000","last_undegraded":"2024-08-15T17:00:29.425813+0000","last_fullsized":"2024-08-15T17:00:29.425813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:28:22.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426700+0000","last_change":"2024-08-15T17:00:29.422214+0000","last_active":"2024-08-15T17:00:29.426700+0000","last_peered":"2024-08-15T17:00:29.426700+0000","last_clean":"2024-08-15T17:00:29.426700+0000","last_became_active":"2024-08-15T17:00:29.421956+0000","last_became_peered":"2024-08-15T17:00:29.421956+0000","last_unstale":"2024-08-15T17:00:29.426700+0000","last_undegraded":"2024-08-15T17:00:29.426700+0000","last_fullsized":"2024-08-15T17:00:29.426700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:21.682359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437544+0000","last_change":"2024-08-15T17:00:29.432980+0000","last_active":"2024-08-15T17:00:29.437544+0000","last_peered":"2024-08-15T17:00:29.437544+0000","last_clean":"2024-08-15T17:00:29.437544+0000","last_became_active":"2024-08-15T17:00:29.432677+0000","last_became_peered":"2024-08-15T17:00:29.432677+0000","last_unstale":"2024-08-15T17:00:29.437544+0000","last_undegraded":"2024-08-15T17:00:29.437544+0000","last_fullsized":"2024-08-15T17:00:29.437544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:41:17.776352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426574+0000","last_change":"2024-08-15T17:00:29.426574+0000","last_active":"2024-08-15T17:00:29.426574+0000","last_peered":"2024-08-15T17:00:29.426574+0000","last_clean":"2024-08-15T17:00:29.426574+0000","last_became_active":"2024-08-15T17:00:29.426351+0000","last_became_peered":"2024-08-15T17:00:29.426351+0000","last_unstale":"2024-08-15T17:00:29.426574+0000","last_undegraded":"2024-08-15T17:00:29.426574+0000","last_fullsized":"2024-08-15T17:00:29.426574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:45:05.108372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425695+0000","last_change":"2024-08-15T17:00:29.425695+0000","last_active":"2024-08-15T17:00:29.425695+0000","last_peered":"2024-08-15T17:00:29.425695+0000","last_clean":"2024-08-15T17:00:29.425695+0000","last_became_active":"2024-08-15T17:00:29.425222+0000","last_became_peered":"2024-08-15T17:00:29.425222+0000","last_unstale":"2024-08-15T17:00:29.425695+0000","last_undegraded":"2024-08-15T17:00:29.425695+0000","last_fullsized":"2024-08-15T17:00:29.425695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:27.346042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426770+0000","last_change":"2024-08-15T17:00:29.426770+0000","last_active":"2024-08-15T17:00:29.426770+0000","last_peered":"2024-08-15T17:00:29.426770+0000","last_clean":"2024-08-15T17:00:29.426770+0000","last_became_active":"2024-08-15T17:00:29.423369+0000","last_became_peered":"2024-08-15T17:00:29.423369+0000","last_unstale":"2024-08-15T17:00:29.426770+0000","last_undegraded":"2024-08-15T17:00:29.426770+0000","last_fullsized":"2024-08-15T17:00:29.426770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:07.974975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423068+0000","last_change":"2024-08-15T17:00:29.423068+0000","last_active":"2024-08-15T17:00:29.423068+0000","last_peered":"2024-08-15T17:00:29.423068+0000","last_clean":"2024-08-15T17:00:29.423068+0000","last_became_active":"2024-08-15T17:00:29.422808+0000","last_became_peered":"2024-08-15T17:00:29.422808+0000","last_unstale":"2024-08-15T17:00:29.423068+0000","last_undegraded":"2024-08-15T17:00:29.423068+0000","last_fullsized":"2024-08-15T17:00:29.423068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:05:37.411070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437762+0000","last_change":"2024-08-15T17:00:28.422039+0000","last_active":"2024-08-15T17:00:29.437762+0000","last_peered":"2024-08-15T17:00:29.437762+0000","last_clean":"2024-08-15T17:00:29.437762+0000","last_became_active":"2024-08-15T17:00:28.421736+0000","last_became_peered":"2024-08-15T17:00:28.421736+0000","last_unstale":"2024-08-15T17:00:29.437762+0000","last_undegraded":"2024-08-15T17:00:29.437762+0000","last_fullsized":"2024-08-15T17:00:29.437762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:00.360870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429335+0000","last_change":"2024-08-15T17:00:29.429335+0000","last_active":"2024-08-15T17:00:29.429335+0000","last_peered":"2024-08-15T17:00:29.429335+0000","last_clean":"2024-08-15T17:00:29.429335+0000","last_became_active":"2024-08-15T17:00:29.429007+0000","last_became_peered":"2024-08-15T17:00:29.429007+0000","last_unstale":"2024-08-15T17:00:29.429335+0000","last_undegraded":"2024-08-15T17:00:29.429335+0000","last_fullsized":"2024-08-15T17:00:29.429335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:46:57.001279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416516+0000","last_change":"2024-08-15T17:00:28.425767+0000","last_active":"2024-08-15T17:00:29.416516+0000","last_peered":"2024-08-15T17:00:29.416516+0000","last_clean":"2024-08-15T17:00:29.416516+0000","last_became_active":"2024-08-15T17:00:28.425429+0000","last_became_peered":"2024-08-15T17:00:28.425429+0000","last_unstale":"2024-08-15T17:00:29.416516+0000","last_undegraded":"2024-08-15T17:00:29.416516+0000","last_fullsized":"2024-08-15T17:00:29.416516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:35:45.386530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437208+0000","last_change":"2024-08-15T17:00:29.432466+0000","last_active":"2024-08-15T17:00:29.437208+0000","last_peered":"2024-08-15T17:00:29.437208+0000","last_clean":"2024-08-15T17:00:29.437208+0000","last_became_active":"2024-08-15T17:00:29.432149+0000","last_became_peered":"2024-08-15T17:00:29.432149+0000","last_unstale":"2024-08-15T17:00:29.437208+0000","last_undegraded":"2024-08-15T17:00:29.437208+0000","last_fullsized":"2024-08-15T17:00:29.437208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:11:56.756218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.447267+0000","last_change":"2024-08-15T17:00:28.429121+0000","last_active":"2024-08-15T17:00:29.447267+0000","last_peered":"2024-08-15T17:00:29.447267+0000","last_clean":"2024-08-15T17:00:29.447267+0000","last_became_active":"2024-08-15T17:00:28.428806+0000","last_became_peered":"2024-08-15T17:00:28.428806+0000","last_unstale":"2024-08-15T17:00:29.447267+0000","last_undegraded":"2024-08-15T17:00:29.447267+0000","last_fullsized":"2024-08-15T17:00:29.447267+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:50:23.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423662+0000","last_change":"2024-08-15T17:00:29.423662+0000","last_active":"2024-08-15T17:00:29.423662+0000","last_peered":"2024-08-15T17:00:29.423662+0000","last_clean":"2024-08-15T17:00:29.423662+0000","last_became_active":"2024-08-15T17:00:29.423373+0000","last_became_peered":"2024-08-15T17:00:29.423373+0000","last_unstale":"2024-08-15T17:00:29.423662+0000","last_undegraded":"2024-08-15T17:00:29.423662+0000","last_fullsized":"2024-08-15T17:00:29.423662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:46:34.342266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.440878+0000","last_change":"2024-08-15T17:00:28.424224+0000","last_active":"2024-08-15T17:00:29.440878+0000","last_peered":"2024-08-15T17:00:29.440878+0000","last_clean":"2024-08-15T17:00:29.440878+0000","last_became_active":"2024-08-15T17:00:28.423938+0000","last_became_peered":"2024-08-15T17:00:28.423938+0000","last_unstale":"2024-08-15T17:00:29.440878+0000","last_undegraded":"2024-08-15T17:00:29.440878+0000","last_fullsized":"2024-08-15T17:00:29.440878+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:16:51.443419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.421401+0000","last_change":"2024-08-15T17:00:29.421401+0000","last_active":"2024-08-15T17:00:29.421401+0000","last_peered":"2024-08-15T17:00:29.421401+0000","last_clean":"2024-08-15T17:00:29.421401+0000","last_became_active":"2024-08-15T17:00:29.421005+0000","last_became_peered":"2024-08-15T17:00:29.421005+0000","last_unstale":"2024-08-15T17:00:29.421401+0000","last_undegraded":"2024-08-15T17:00:29.421401+0000","last_fullsized":"2024-08-15T17:00:29.421401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:18:17.891660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.442092+0000","last_change":"2024-08-15T17:00:28.423471+0000","last_active":"2024-08-15T17:00:29.442092+0000","last_peered":"2024-08-15T17:00:29.442092+0000","last_clean":"2024-08-15T17:00:29.442092+0000","last_became_active":"2024-08-15T17:00:28.422972+0000","last_became_peered":"2024-08-15T17:00:28.422972+0000","last_unstale":"2024-08-15T17:00:29.442092+0000","last_undegraded":"2024-08-15T17:00:29.442092+0000","last_fullsized":"2024-08-15T17:00:29.442092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:05:02.388046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429924+0000","last_change":"2024-08-15T17:00:29.427530+0000","last_active":"2024-08-15T17:00:29.429924+0000","last_peered":"2024-08-15T17:00:29.429924+0000","last_clean":"2024-08-15T17:00:29.429924+0000","last_became_active":"2024-08-15T17:00:29.427256+0000","last_became_peered":"2024-08-15T17:00:29.427256+0000","last_unstale":"2024-08-15T17:00:29.429924+0000","last_undegraded":"2024-08-15T17:00:29.429924+0000","last_fullsized":"2024-08-15T17:00:29.429924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:03:13.551921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.445851+0000","last_change":"2024-08-15T17:00:28.423304+0000","last_active":"2024-08-15T17:00:29.445851+0000","last_peered":"2024-08-15T17:00:29.445851+0000","last_clean":"2024-08-15T17:00:29.445851+0000","last_became_active":"2024-08-15T17:00:28.422986+0000","last_became_peered":"2024-08-15T17:00:28.422986+0000","last_unstale":"2024-08-15T17:00:29.445851+0000","last_undegraded":"2024-08-15T17:00:29.445851+0000","last_fullsized":"2024-08-15T17:00:29.445851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:59.476023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430188+0000","last_change":"2024-08-15T17:00:29.424259+0000","last_active":"2024-08-15T17:00:29.430188+0000","last_peered":"2024-08-15T17:00:29.430188+0000","last_clean":"2024-08-15T17:00:29.430188+0000","last_became_active":"2024-08-15T17:00:29.423818+0000","last_became_peered":"2024-08-15T17:00:29.423818+0000","last_unstale":"2024-08-15T17:00:29.430188+0000","last_undegraded":"2024-08-15T17:00:29.430188+0000","last_fullsized":"2024-08-15T17:00:29.430188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:28:54.704212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.426356+0000","last_change":"2024-08-15T17:00:28.422923+0000","last_active":"2024-08-15T17:00:28.426356+0000","last_peered":"2024-08-15T17:00:28.426356+0000","last_clean":"2024-08-15T17:00:28.426356+0000","last_became_active":"2024-08-15T17:00:28.422471+0000","last_became_peered":"2024-08-15T17:00:28.422471+0000","last_unstale":"2024-08-15T17:00:28.426356+0000","last_undegraded":"2024-08-15T17:00:28.426356+0000","last_fullsized":"2024-08-15T17:00:28.426356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:01.393488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416623+0000","last_change":"2024-08-15T17:00:29.416623+0000","last_active":"2024-08-15T17:00:29.416623+0000","last_peered":"2024-08-15T17:00:29.416623+0000","last_clean":"2024-08-15T17:00:29.416623+0000","last_became_active":"2024-08-15T17:00:29.415765+0000","last_became_peered":"2024-08-15T17:00:29.415765+0000","last_unstale":"2024-08-15T17:00:29.416623+0000","last_undegraded":"2024-08-15T17:00:29.416623+0000","last_fullsized":"2024-08-15T17:00:29.416623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:28:26.456498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416767+0000","last_change":"2024-08-15T17:00:28.419871+0000","last_active":"2024-08-15T17:00:29.416767+0000","last_peered":"2024-08-15T17:00:29.416767+0000","last_clean":"2024-08-15T17:00:29.416767+0000","last_became_active":"2024-08-15T17:00:28.419568+0000","last_became_peered":"2024-08-15T17:00:28.419568+0000","last_unstale":"2024-08-15T17:00:29.416767+0000","last_undegraded":"2024-08-15T17:00:29.416767+0000","last_fullsized":"2024-08-15T17:00:29.416767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:54:20.081307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.436887+0000","last_change":"2024-08-15T17:00:29.433377+0000","last_active":"2024-08-15T17:00:29.436887+0000","last_peered":"2024-08-15T17:00:29.436887+0000","last_clean":"2024-08-15T17:00:29.436887+0000","last_became_active":"2024-08-15T17:00:29.433110+0000","last_became_peered":"2024-08-15T17:00:29.433110+0000","last_unstale":"2024-08-15T17:00:29.436887+0000","last_undegraded":"2024-08-15T17:00:29.436887+0000","last_fullsized":"2024-08-15T17:00:29.436887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:45:53.355016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429272+0000","last_change":"2024-08-15T17:00:28.427147+0000","last_active":"2024-08-15T17:00:28.429272+0000","last_peered":"2024-08-15T17:00:28.429272+0000","last_clean":"2024-08-15T17:00:28.429272+0000","last_became_active":"2024-08-15T17:00:28.426821+0000","last_became_peered":"2024-08-15T17:00:28.426821+0000","last_unstale":"2024-08-15T17:00:28.429272+0000","last_undegraded":"2024-08-15T17:00:28.429272+0000","last_fullsized":"2024-08-15T17:00:28.429272+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:19:19.327077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429298+0000","last_change":"2024-08-15T17:00:29.429298+0000","last_active":"2024-08-15T17:00:29.429298+0000","last_peered":"2024-08-15T17:00:29.429298+0000","last_clean":"2024-08-15T17:00:29.429298+0000","last_became_active":"2024-08-15T17:00:29.428959+0000","last_became_peered":"2024-08-15T17:00:29.428959+0000","last_unstale":"2024-08-15T17:00:29.429298+0000","last_undegraded":"2024-08-15T17:00:29.429298+0000","last_fullsized":"2024-08-15T17:00:29.429298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:06.118124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.424041+0000","last_change":"2024-08-15T17:00:28.423122+0000","last_active":"2024-08-15T17:00:28.424041+0000","last_peered":"2024-08-15T17:00:28.424041+0000","last_clean":"2024-08-15T17:00:28.424041+0000","last_became_active":"2024-08-15T17:00:28.422810+0000","last_became_peered":"2024-08-15T17:00:28.422810+0000","last_unstale":"2024-08-15T17:00:28.424041+0000","last_undegraded":"2024-08-15T17:00:28.424041+0000","last_fullsized":"2024-08-15T17:00:28.424041+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:43.420539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431800+0000","last_change":"2024-08-15T17:00:29.431800+0000","last_active":"2024-08-15T17:00:29.431800+0000","last_peered":"2024-08-15T17:00:29.431800+0000","last_clean":"2024-08-15T17:00:29.431800+0000","last_became_active":"2024-08-15T17:00:29.430099+0000","last_became_peered":"2024-08-15T17:00:29.430099+0000","last_unstale":"2024-08-15T17:00:29.431800+0000","last_undegraded":"2024-08-15T17:00:29.431800+0000","last_fullsized":"2024-08-15T17:00:29.431800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:48.679569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437603+0000","last_change":"2024-08-15T17:00:28.424682+0000","last_active":"2024-08-15T17:00:29.437603+0000","last_peered":"2024-08-15T17:00:29.437603+0000","last_clean":"2024-08-15T17:00:29.437603+0000","last_became_active":"2024-08-15T17:00:28.424235+0000","last_became_peered":"2024-08-15T17:00:28.424235+0000","last_unstale":"2024-08-15T17:00:29.437603+0000","last_undegraded":"2024-08-15T17:00:29.437603+0000","last_fullsized":"2024-08-15T17:00:29.437603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:04:49.362272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429932+0000","last_change":"2024-08-15T17:00:29.426875+0000","last_active":"2024-08-15T17:00:29.429932+0000","last_peered":"2024-08-15T17:00:29.429932+0000","last_clean":"2024-08-15T17:00:29.429932+0000","last_became_active":"2024-08-15T17:00:29.425983+0000","last_became_peered":"2024-08-15T17:00:29.425983+0000","last_unstale":"2024-08-15T17:00:29.429932+0000","last_undegraded":"2024-08-15T17:00:29.429932+0000","last_fullsized":"2024-08-15T17:00:29.429932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:57:53.772660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.447874+0000","last_change":"2024-08-15T17:00:28.425753+0000","last_active":"2024-08-15T17:00:29.447874+0000","last_peered":"2024-08-15T17:00:29.447874+0000","last_clean":"2024-08-15T17:00:29.447874+0000","last_became_active":"2024-08-15T17:00:28.425178+0000","last_became_peered":"2024-08-15T17:00:28.425178+0000","last_unstale":"2024-08-15T17:00:29.447874+0000","last_undegraded":"2024-08-15T17:00:29.447874+0000","last_fullsized":"2024-08-15T17:00:29.447874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:46:55.995987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426795+0000","last_change":"2024-08-15T17:00:29.426795+0000","last_active":"2024-08-15T17:00:29.426795+0000","last_peered":"2024-08-15T17:00:29.426795+0000","last_clean":"2024-08-15T17:00:29.426795+0000","last_became_active":"2024-08-15T17:00:29.426452+0000","last_became_peered":"2024-08-15T17:00:29.426452+0000","last_unstale":"2024-08-15T17:00:29.426795+0000","last_undegraded":"2024-08-15T17:00:29.426795+0000","last_fullsized":"2024-08-15T17:00:29.426795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:38:05.927974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439253+0000","last_change":"2024-08-15T17:00:28.420058+0000","last_active":"2024-08-15T17:00:29.439253+0000","last_peered":"2024-08-15T17:00:29.439253+0000","last_clean":"2024-08-15T17:00:29.439253+0000","last_became_active":"2024-08-15T17:00:28.419386+0000","last_became_peered":"2024-08-15T17:00:28.419386+0000","last_unstale":"2024-08-15T17:00:29.439253+0000","last_undegraded":"2024-08-15T17:00:29.439253+0000","last_fullsized":"2024-08-15T17:00:29.439253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:13:30.688015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429274+0000","last_change":"2024-08-15T17:00:29.429274+0000","last_active":"2024-08-15T17:00:29.429274+0000","last_peered":"2024-08-15T17:00:29.429274+0000","last_clean":"2024-08-15T17:00:29.429274+0000","last_became_active":"2024-08-15T17:00:29.428695+0000","last_became_peered":"2024-08-15T17:00:29.428695+0000","last_unstale":"2024-08-15T17:00:29.429274+0000","last_undegraded":"2024-08-15T17:00:29.429274+0000","last_fullsized":"2024-08-15T17:00:29.429274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:49:07.734249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.427807+0000","last_change":"2024-08-15T17:00:28.424742+0000","last_active":"2024-08-15T17:00:28.427807+0000","last_peered":"2024-08-15T17:00:28.427807+0000","last_clean":"2024-08-15T17:00:28.427807+0000","last_became_active":"2024-08-15T17:00:28.424410+0000","last_became_peered":"2024-08-15T17:00:28.424410+0000","last_unstale":"2024-08-15T17:00:28.427807+0000","last_undegraded":"2024-08-15T17:00:28.427807+0000","last_fullsized":"2024-08-15T17:00:28.427807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:36:19.262350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425018+0000","last_change":"2024-08-15T17:00:29.425018+0000","last_active":"2024-08-15T17:00:29.425018+0000","last_peered":"2024-08-15T17:00:29.425018+0000","last_clean":"2024-08-15T17:00:29.425018+0000","last_became_active":"2024-08-15T17:00:29.423702+0000","last_became_peered":"2024-08-15T17:00:29.423702+0000","last_unstale":"2024-08-15T17:00:29.425018+0000","last_undegraded":"2024-08-15T17:00:29.425018+0000","last_fullsized":"2024-08-15T17:00:29.425018+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.426376+0000","last_change":"2024-08-15T17:00:28.423019+0000","last_active":"2024-08-15T17:00:28.426376+0000","last_peered":"2024-08-15T17:00:28.426376+0000","last_clean":"2024-08-15T17:00:28.426376+0000","last_became_active":"2024-08-15T17:00:28.422097+0000","last_became_peered":"2024-08-15T17:00:28.422097+0000","last_unstale":"2024-08-15T17:00:28.426376+0000","last_undegraded":"2024-08-15T17:00:28.426376+0000","last_fullsized":"2024-08-15T17:00:28.426376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:25.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423544+0000","last_change":"2024-08-15T17:00:29.423544+0000","last_active":"2024-08-15T17:00:29.423544+0000","last_peered":"2024-08-15T17:00:29.423544+0000","last_clean":"2024-08-15T17:00:29.423544+0000","last_became_active":"2024-08-15T17:00:29.423275+0000","last_became_peered":"2024-08-15T17:00:29.423275+0000","last_unstale":"2024-08-15T17:00:29.423544+0000","last_undegraded":"2024-08-15T17:00:29.423544+0000","last_fullsized":"2024-08-15T17:00:29.423544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:02.398781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.448558+0000","last_change":"2024-08-15T17:00:28.428441+0000","last_active":"2024-08-15T17:00:29.448558+0000","last_peered":"2024-08-15T17:00:29.448558+0000","last_clean":"2024-08-15T17:00:29.448558+0000","last_became_active":"2024-08-15T17:00:28.428136+0000","last_became_peered":"2024-08-15T17:00:28.428136+0000","last_unstale":"2024-08-15T17:00:29.448558+0000","last_undegraded":"2024-08-15T17:00:29.448558+0000","last_fullsized":"2024-08-15T17:00:29.448558+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:26.340750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422610+0000","last_change":"2024-08-15T17:00:29.419470+0000","last_active":"2024-08-15T17:00:29.422610+0000","last_peered":"2024-08-15T17:00:29.422610+0000","last_clean":"2024-08-15T17:00:29.422610+0000","last_became_active":"2024-08-15T17:00:29.419187+0000","last_became_peered":"2024-08-15T17:00:29.419187+0000","last_unstale":"2024-08-15T17:00:29.422610+0000","last_undegraded":"2024-08-15T17:00:29.422610+0000","last_fullsized":"2024-08-15T17:00:29.422610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:39:49.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.419541+0000","last_change":"2024-08-15T17:00:28.423384+0000","last_active":"2024-08-15T17:00:29.419541+0000","last_peered":"2024-08-15T17:00:29.419541+0000","last_clean":"2024-08-15T17:00:29.419541+0000","last_became_active":"2024-08-15T17:00:28.423050+0000","last_became_peered":"2024-08-15T17:00:28.423050+0000","last_unstale":"2024-08-15T17:00:29.419541+0000","last_undegraded":"2024-08-15T17:00:29.419541+0000","last_fullsized":"2024-08-15T17:00:29.419541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:00.360870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.428870+0000","last_change":"2024-08-15T17:00:29.424989+0000","last_active":"2024-08-15T17:00:29.428870+0000","last_peered":"2024-08-15T17:00:29.428870+0000","last_clean":"2024-08-15T17:00:29.428870+0000","last_became_active":"2024-08-15T17:00:29.424715+0000","last_became_peered":"2024-08-15T17:00:29.424715+0000","last_unstale":"2024-08-15T17:00:29.428870+0000","last_undegraded":"2024-08-15T17:00:29.428870+0000","last_fullsized":"2024-08-15T17:00:29.428870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:07:50.195961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439149+0000","last_change":"2024-08-15T17:00:28.418886+0000","last_active":"2024-08-15T17:00:29.439149+0000","last_peered":"2024-08-15T17:00:29.439149+0000","last_clean":"2024-08-15T17:00:29.439149+0000","last_became_active":"2024-08-15T17:00:28.418589+0000","last_became_peered":"2024-08-15T17:00:28.418589+0000","last_unstale":"2024-08-15T17:00:29.439149+0000","last_undegraded":"2024-08-15T17:00:29.439149+0000","last_fullsized":"2024-08-15T17:00:29.439149+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:43.420539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423612+0000","last_change":"2024-08-15T17:00:29.423612+0000","last_active":"2024-08-15T17:00:29.423612+0000","last_peered":"2024-08-15T17:00:29.423612+0000","last_clean":"2024-08-15T17:00:29.423612+0000","last_became_active":"2024-08-15T17:00:29.423257+0000","last_became_peered":"2024-08-15T17:00:29.423257+0000","last_unstale":"2024-08-15T17:00:29.423612+0000","last_undegraded":"2024-08-15T17:00:29.423612+0000","last_fullsized":"2024-08-15T17:00:29.423612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:20:29.415382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439226+0000","last_change":"2024-08-15T17:00:28.417723+0000","last_active":"2024-08-15T17:00:29.439226+0000","last_peered":"2024-08-15T17:00:29.439226+0000","last_clean":"2024-08-15T17:00:29.439226+0000","last_became_active":"2024-08-15T17:00:28.417299+0000","last_became_peered":"2024-08-15T17:00:28.417299+0000","last_unstale":"2024-08-15T17:00:29.439226+0000","last_undegraded":"2024-08-15T17:00:29.439226+0000","last_fullsized":"2024-08-15T17:00:29.439226+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:44:51.570213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.424585+0000","last_change":"2024-08-15T17:00:29.422313+0000","last_active":"2024-08-15T17:00:29.424585+0000","last_peered":"2024-08-15T17:00:29.424585+0000","last_clean":"2024-08-15T17:00:29.424585+0000","last_became_active":"2024-08-15T17:00:29.422019+0000","last_became_peered":"2024-08-15T17:00:29.422019+0000","last_unstale":"2024-08-15T17:00:29.424585+0000","last_undegraded":"2024-08-15T17:00:29.424585+0000","last_fullsized":"2024-08-15T17:00:29.424585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:17:46.929433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.412637+0000","last_change":"2024-08-15T17:00:28.423106+0000","last_active":"2024-08-15T17:00:29.412637+0000","last_peered":"2024-08-15T17:00:29.412637+0000","last_clean":"2024-08-15T17:00:29.412637+0000","last_became_active":"2024-08-15T17:00:28.422571+0000","last_became_peered":"2024-08-15T17:00:28.422571+0000","last_unstale":"2024-08-15T17:00:29.412637+0000","last_undegraded":"2024-08-15T17:00:29.412637+0000","last_fullsized":"2024-08-15T17:00:29.412637+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:51:15.739317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423326+0000","last_change":"2024-08-15T17:00:29.422070+0000","last_active":"2024-08-15T17:00:29.423326+0000","last_peered":"2024-08-15T17:00:29.423326+0000","last_clean":"2024-08-15T17:00:29.423326+0000","last_became_active":"2024-08-15T17:00:29.421375+0000","last_became_peered":"2024-08-15T17:00:29.421375+0000","last_unstale":"2024-08-15T17:00:29.423326+0000","last_undegraded":"2024-08-15T17:00:29.423326+0000","last_fullsized":"2024-08-15T17:00:29.423326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:52:38.910591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437006+0000","last_change":"2024-08-15T17:00:28.426143+0000","last_active":"2024-08-15T17:00:29.437006+0000","last_peered":"2024-08-15T17:00:29.437006+0000","last_clean":"2024-08-15T17:00:29.437006+0000","last_became_active":"2024-08-15T17:00:28.425808+0000","last_became_peered":"2024-08-15T17:00:28.425808+0000","last_unstale":"2024-08-15T17:00:29.437006+0000","last_undegraded":"2024-08-15T17:00:29.437006+0000","last_fullsized":"2024-08-15T17:00:29.437006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:44:51.570213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431433+0000","last_change":"2024-08-15T17:00:29.428735+0000","last_active":"2024-08-15T17:00:29.431433+0000","last_peered":"2024-08-15T17:00:29.431433+0000","last_clean":"2024-08-15T17:00:29.431433+0000","last_became_active":"2024-08-15T17:00:29.427893+0000","last_became_peered":"2024-08-15T17:00:29.427893+0000","last_unstale":"2024-08-15T17:00:29.431433+0000","last_undegraded":"2024-08-15T17:00:29.431433+0000","last_fullsized":"2024-08-15T17:00:29.431433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:26.774890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429086+0000","last_change":"2024-08-15T17:00:28.425736+0000","last_active":"2024-08-15T17:00:28.429086+0000","last_peered":"2024-08-15T17:00:28.429086+0000","last_clean":"2024-08-15T17:00:28.429086+0000","last_became_active":"2024-08-15T17:00:28.425410+0000","last_became_peered":"2024-08-15T17:00:28.425410+0000","last_unstale":"2024-08-15T17:00:28.429086+0000","last_undegraded":"2024-08-15T17:00:28.429086+0000","last_fullsized":"2024-08-15T17:00:28.429086+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:25.769597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430469+0000","last_change":"2024-08-15T17:00:29.427385+0000","last_active":"2024-08-15T17:00:29.430469+0000","last_peered":"2024-08-15T17:00:29.430469+0000","last_clean":"2024-08-15T17:00:29.430469+0000","last_became_active":"2024-08-15T17:00:29.427067+0000","last_became_peered":"2024-08-15T17:00:29.427067+0000","last_unstale":"2024-08-15T17:00:29.430469+0000","last_undegraded":"2024-08-15T17:00:29.430469+0000","last_fullsized":"2024-08-15T17:00:29.430469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:01.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.411756+0000","last_change":"2024-08-15T17:00:28.423412+0000","last_active":"2024-08-15T17:00:29.411756+0000","last_peered":"2024-08-15T17:00:29.411756+0000","last_clean":"2024-08-15T17:00:29.411756+0000","last_became_active":"2024-08-15T17:00:28.423160+0000","last_became_peered":"2024-08-15T17:00:28.423160+0000","last_unstale":"2024-08-15T17:00:29.411756+0000","last_undegraded":"2024-08-15T17:00:29.411756+0000","last_fullsized":"2024-08-15T17:00:29.411756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:45:14.327141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425911+0000","last_change":"2024-08-15T17:00:29.425911+0000","last_active":"2024-08-15T17:00:29.425911+0000","last_peered":"2024-08-15T17:00:29.425911+0000","last_clean":"2024-08-15T17:00:29.425911+0000","last_became_active":"2024-08-15T17:00:29.425522+0000","last_became_peered":"2024-08-15T17:00:29.425522+0000","last_unstale":"2024-08-15T17:00:29.425911+0000","last_undegraded":"2024-08-15T17:00:29.425911+0000","last_fullsized":"2024-08-15T17:00:29.425911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:28:22.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.412891+0000","last_change":"2024-08-15T17:00:28.425065+0000","last_active":"2024-08-15T17:00:29.412891+0000","last_peered":"2024-08-15T17:00:29.412891+0000","last_clean":"2024-08-15T17:00:29.412891+0000","last_became_active":"2024-08-15T17:00:28.424314+0000","last_became_peered":"2024-08-15T17:00:28.424314+0000","last_unstale":"2024-08-15T17:00:29.412891+0000","last_undegraded":"2024-08-15T17:00:29.412891+0000","last_fullsized":"2024-08-15T17:00:29.412891+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:42.488167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422760+0000","last_change":"2024-08-15T17:00:29.419198+0000","last_active":"2024-08-15T17:00:29.422760+0000","last_peered":"2024-08-15T17:00:29.422760+0000","last_clean":"2024-08-15T17:00:29.422760+0000","last_became_active":"2024-08-15T17:00:29.418831+0000","last_became_peered":"2024-08-15T17:00:29.418831+0000","last_unstale":"2024-08-15T17:00:29.422760+0000","last_undegraded":"2024-08-15T17:00:29.422760+0000","last_fullsized":"2024-08-15T17:00:29.422760+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:51:18.474310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.440400+0000","last_change":"2024-08-15T17:00:28.424578+0000","last_active":"2024-08-15T17:00:29.440400+0000","last_peered":"2024-08-15T17:00:29.440400+0000","last_clean":"2024-08-15T17:00:29.440400+0000","last_became_active":"2024-08-15T17:00:28.424252+0000","last_became_peered":"2024-08-15T17:00:28.424252+0000","last_unstale":"2024-08-15T17:00:29.440400+0000","last_undegraded":"2024-08-15T17:00:29.440400+0000","last_fullsized":"2024-08-15T17:00:29.440400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:17:45.924140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430294+0000","last_change":"2024-08-15T17:00:29.428679+0000","last_active":"2024-08-15T17:00:29.430294+0000","last_peered":"2024-08-15T17:00:29.430294+0000","last_clean":"2024-08-15T17:00:29.430294+0000","last_became_active":"2024-08-15T17:00:29.428394+0000","last_became_peered":"2024-08-15T17:00:29.428394+0000","last_unstale":"2024-08-15T17:00:29.430294+0000","last_undegraded":"2024-08-15T17:00:29.430294+0000","last_fullsized":"2024-08-15T17:00:29.430294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:24:32.992039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.428506+0000","last_change":"2024-08-15T17:00:28.426296+0000","last_active":"2024-08-15T17:00:28.428506+0000","last_peered":"2024-08-15T17:00:28.428506+0000","last_clean":"2024-08-15T17:00:28.428506+0000","last_became_active":"2024-08-15T17:00:28.425999+0000","last_became_peered":"2024-08-15T17:00:28.425999+0000","last_unstale":"2024-08-15T17:00:28.428506+0000","last_undegraded":"2024-08-15T17:00:28.428506+0000","last_fullsized":"2024-08-15T17:00:28.428506+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:13:30.688015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431228+0000","last_change":"2024-08-15T17:00:29.431228+0000","last_active":"2024-08-15T17:00:29.431228+0000","last_peered":"2024-08-15T17:00:29.431228+0000","last_clean":"2024-08-15T17:00:29.431228+0000","last_became_active":"2024-08-15T17:00:29.430931+0000","last_became_peered":"2024-08-15T17:00:29.430931+0000","last_unstale":"2024-08-15T17:00:29.431228+0000","last_undegraded":"2024-08-15T17:00:29.431228+0000","last_fullsized":"2024-08-15T17:00:29.431228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:39:49.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.425124+0000","last_change":"2024-08-15T17:00:28.418354+0000","last_active":"2024-08-15T17:00:28.425124+0000","last_peered":"2024-08-15T17:00:28.425124+0000","last_clean":"2024-08-15T17:00:28.425124+0000","last_became_active":"2024-08-15T17:00:28.418012+0000","last_became_peered":"2024-08-15T17:00:28.418012+0000","last_unstale":"2024-08-15T17:00:28.425124+0000","last_undegraded":"2024-08-15T17:00:28.425124+0000","last_fullsized":"2024-08-15T17:00:28.425124+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:30.678372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.427791+0000","last_change":"2024-08-15T17:00:29.427791+0000","last_active":"2024-08-15T17:00:29.427791+0000","last_peered":"2024-08-15T17:00:29.427791+0000","last_clean":"2024-08-15T17:00:29.427791+0000","last_became_active":"2024-08-15T17:00:29.427521+0000","last_became_peered":"2024-08-15T17:00:29.427521+0000","last_unstale":"2024-08-15T17:00:29.427791+0000","last_undegraded":"2024-08-15T17:00:29.427791+0000","last_fullsized":"2024-08-15T17:00:29.427791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:20:29.415382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.424959+0000","last_change":"2024-08-15T17:00:28.424509+0000","last_active":"2024-08-15T17:00:28.424959+0000","last_peered":"2024-08-15T17:00:28.424959+0000","last_clean":"2024-08-15T17:00:28.424959+0000","last_became_active":"2024-08-15T17:00:28.423796+0000","last_became_peered":"2024-08-15T17:00:28.423796+0000","last_unstale":"2024-08-15T17:00:28.424959+0000","last_undegraded":"2024-08-15T17:00:28.424959+0000","last_fullsized":"2024-08-15T17:00:28.424959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:59.476023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.427684+0000","last_change":"2024-08-15T17:00:29.427684+0000","last_active":"2024-08-15T17:00:29.427684+0000","last_peered":"2024-08-15T17:00:29.427684+0000","last_clean":"2024-08-15T17:00:29.427684+0000","last_became_active":"2024-08-15T17:00:29.427060+0000","last_became_peered":"2024-08-15T17:00:29.427060+0000","last_unstale":"2024-08-15T17:00:29.427684+0000","last_undegraded":"2024-08-15T17:00:29.427684+0000","last_fullsized":"2024-08-15T17:00:29.427684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:59:20.756910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.427648+0000","last_change":"2024-08-15T17:00:28.422373+0000","last_active":"2024-08-15T17:00:28.427648+0000","last_peered":"2024-08-15T17:00:28.427648+0000","last_clean":"2024-08-15T17:00:28.427648+0000","last_became_active":"2024-08-15T17:00:28.420142+0000","last_became_peered":"2024-08-15T17:00:28.420142+0000","last_unstale":"2024-08-15T17:00:28.427648+0000","last_undegraded":"2024-08-15T17:00:28.427648+0000","last_fullsized":"2024-08-15T17:00:28.427648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:25.769597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.433127+0000","last_change":"2024-08-15T17:00:29.433127+0000","last_active":"2024-08-15T17:00:29.433127+0000","last_peered":"2024-08-15T17:00:29.433127+0000","last_clean":"2024-08-15T17:00:29.433127+0000","last_became_active":"2024-08-15T17:00:29.432696+0000","last_became_peered":"2024-08-15T17:00:29.432696+0000","last_unstale":"2024-08-15T17:00:29.433127+0000","last_undegraded":"2024-08-15T17:00:29.433127+0000","last_fullsized":"2024-08-15T17:00:29.433127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:19:21.726365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.411869+0000","last_change":"2024-08-15T17:00:28.419753+0000","last_active":"2024-08-15T17:00:29.411869+0000","last_peered":"2024-08-15T17:00:29.411869+0000","last_clean":"2024-08-15T17:00:29.411869+0000","last_became_active":"2024-08-15T17:00:28.419472+0000","last_became_peered":"2024-08-15T17:00:28.419472+0000","last_unstale":"2024-08-15T17:00:29.411869+0000","last_undegraded":"2024-08-15T17:00:29.411869+0000","last_fullsized":"2024-08-15T17:00:29.411869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:59.291885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437305+0000","last_change":"2024-08-15T17:00:29.432941+0000","last_active":"2024-08-15T17:00:29.437305+0000","last_peered":"2024-08-15T17:00:29.437305+0000","last_clean":"2024-08-15T17:00:29.437305+0000","last_became_active":"2024-08-15T17:00:29.432650+0000","last_became_peered":"2024-08-15T17:00:29.432650+0000","last_unstale":"2024-08-15T17:00:29.437305+0000","last_undegraded":"2024-08-15T17:00:29.437305+0000","last_fullsized":"2024-08-15T17:00:29.437305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:49:16.516093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429039+0000","last_change":"2024-08-15T17:00:28.427297+0000","last_active":"2024-08-15T17:00:28.429039+0000","last_peered":"2024-08-15T17:00:28.429039+0000","last_clean":"2024-08-15T17:00:28.429039+0000","last_became_active":"2024-08-15T17:00:28.426965+0000","last_became_peered":"2024-08-15T17:00:28.426965+0000","last_unstale":"2024-08-15T17:00:28.429039+0000","last_undegraded":"2024-08-15T17:00:28.429039+0000","last_fullsized":"2024-08-15T17:00:28.429039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:24.126020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422540+0000","last_change":"2024-08-15T17:00:29.422540+0000","last_active":"2024-08-15T17:00:29.422540+0000","last_peered":"2024-08-15T17:00:29.422540+0000","last_clean":"2024-08-15T17:00:29.422540+0000","last_became_active":"2024-08-15T17:00:29.422228+0000","last_became_peered":"2024-08-15T17:00:29.422228+0000","last_unstale":"2024-08-15T17:00:29.422540+0000","last_undegraded":"2024-08-15T17:00:29.422540+0000","last_fullsized":"2024-08-15T17:00:29.422540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:42:00.898538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.423804+0000","last_change":"2024-08-15T17:00:28.420779+0000","last_active":"2024-08-15T17:00:28.423804+0000","last_peered":"2024-08-15T17:00:28.423804+0000","last_clean":"2024-08-15T17:00:28.423804+0000","last_became_active":"2024-08-15T17:00:28.420422+0000","last_became_peered":"2024-08-15T17:00:28.420422+0000","last_unstale":"2024-08-15T17:00:28.423804+0000","last_undegraded":"2024-08-15T17:00:28.423804+0000","last_fullsized":"2024-08-15T17:00:28.423804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:30.678372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.416883+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:00:28.416883+0000","last_peered":"2024-08-15T17:00:28.416883+0000","last_clean":"2024-08-15T17:00:28.416883+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:00:28.416883+0000","last_undegraded":"2024-08-15T17:00:28.416883+0000","last_fullsized":"2024-08-15T17:00:28.416883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":942080,"data_stored":721019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":352256,"data_stored":131165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":364544,"data_stored":137509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":352256,"data_stored":130665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":360448,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":360448,"data_stored":136919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":360448,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":933888,"data_stored":719968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:00:33.470 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:00:33.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.569+0000 7fb1b2130700 1 Processor -- start 2024-08-15T17:00:33.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.569+0000 7fb1b2130700 1 -- start start 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac155d80 0x7fb1ac0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb1ac0af420 0x7fb1ac0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ac0af850 con 0x7fb1ac0ad090 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0ad8a0 con 0x7fb1ac0af420 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac0adb80 con 0x7fb1ac155d80 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb1ac0af420 0x7fb1ac0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb1ac0af420 0x7fb1ac0acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50936/0 (socket says 172.21.15.146:50936) 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 -- 172.21.15.146:0/765208653 learned_addr learned my addr 172.21.15.146:0/765208653 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:33.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b112e700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac155d80 0x7fb1ac0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 -- 172.21.15.146:0/765208653 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac155d80 msgr2=0x7fb1ac0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:33.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac155d80 0x7fb1ac0aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 -- 172.21.15.146:0/765208653 >> v1:172.21.15.161:6789/0 conn(0x7fb1ac0ad090 legacy=0x7fb1ac1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 -- 172.21.15.146:0/765208653 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac0ade60 con 0x7fb1ac0af420 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b092d700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac0af420 0x7fb1ac0acb50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0099e0 tx=0x7fb19c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c55ecf92934b38 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1a3fff700 1 -- 172.21.15.146:0/765208653 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00b820 con 0x7fb1ac0af420 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1a3fff700 1 -- 172.21.15.146:0/765208653 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b980 con 0x7fb1ac0af420 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1a3fff700 1 -- 172.21.15.146:0/765208653 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c00bd20 con 0x7fb1ac0af420 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- 172.21.15.146:0/765208653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac0af420 msgr2=0x7fb1ac0acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac0af420 0x7fb1ac0acb50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0099e0 tx=0x7fb19c009250 comp rx=0 tx=0).stop 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- 172.21.15.146:0/765208653 shutdown_connections 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac0af420 0x7fb1ac0acb50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 --2- 172.21.15.146:0/765208653 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac155d80 0x7fb1ac0aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- 172.21.15.146:0/765208653 >> 172.21.15.146:0/765208653 conn(0x7fb1ac06e070 msgr2=0x7fb1ac0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- 172.21.15.146:0/765208653 shutdown_connections 2024-08-15T17:00:33.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.573+0000 7fb1b2130700 1 -- 172.21.15.146:0/765208653 wait complete. 2024-08-15T17:00:33.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 Processor -- start 2024-08-15T17:00:33.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 -- start start 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac0ad090 0x7fb1ac06ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1ac0af420 0x7fb1ac06b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14bf50 con 0x7fb1ac155d80 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14bac0 con 0x7fb1ac0af420 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b2130700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14aeb0 con 0x7fb1ac0ad090 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b192f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b192f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50950/0 (socket says 172.21.15.146:50950) 2024-08-15T17:00:33.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b192f700 1 -- 172.21.15.146:0/363280814 learned_addr learned my addr 172.21.15.146:0/363280814 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b112e700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac0ad090 0x7fb1ac06ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.577+0000 7fb1b092d700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1ac0af420 0x7fb1ac06b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 -- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac0ad090 msgr2=0x7fb1ac06ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac0ad090 0x7fb1ac06ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 -- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1ac0af420 msgr2=0x7fb1ac06b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1ac0af420 0x7fb1ac06b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 -- 172.21.15.146:0/363280814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c0094f0 con 0x7fb1ac155d80 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b192f700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c8e0 tx=0x7fb1a4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4012630 con 0x7fb1ac155d80 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4012790 con 0x7fb1ac155d80 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac14b190 con 0x7fb1ac155d80 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4012b50 con 0x7fb1ac155d80 2024-08-15T17:00:33.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac14b750 con 0x7fb1ac155d80 2024-08-15T17:00:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac14bc30 con 0x7fb1ac155d80 2024-08-15T17:00:33.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb1a401a3a0 con 0x7fb1ac155d80 2024-08-15T17:00:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 0x7fb190062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb1a407cd20 con 0x7fb1ac155d80 2024-08-15T17:00:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.581+0000 7fb1b112e700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 0x7fb190062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.585+0000 7fb1b112e700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 0x7fb190062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009d10 tx=0x7fb1a80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.589+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb1a404d170 con 0x7fb1ac155d80 2024-08-15T17:00:33.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.753+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1ac0638f0 con 0x7fb190060860 2024-08-15T17:00:33.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.769+0000 7fb1a17fa700 1 -- 172.21.15.146:0/363280814 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264526 (secure 0 0 0) 0x7fb1ac0638f0 con 0x7fb190060860 2024-08-15T17:00:33.786 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 msgr2=0x7fb190062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 0x7fb190062d10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009d10 tx=0x7fb1a80091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 msgr2=0x7fb1ac14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400c8e0 tx=0x7fb1a4001710 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 shutdown_connections 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb190060860 0x7fb190062d10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb1ac155d80 0x7fb1ac14a180 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb1ac0af420 0x7fb1ac06b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 --2- 172.21.15.146:0/363280814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb1ac0ad090 0x7fb1ac06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 >> 172.21.15.146:0/363280814 conn(0x7fb1ac06e070 msgr2=0x7fb1ac0b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 shutdown_connections 2024-08-15T17:00:33.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.777+0000 7fb1b2130700 1 -- 172.21.15.146:0/363280814 wait complete. 2024-08-15T17:00:33.790 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:00:33.803 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-08-15T17:00:33.661364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2386968,"kb_used_data":3932,"kb_used_omap":13,"kb_used_meta":2382898,"kb_avail":752587752,"statfs":{"total":773094113280,"available":770649858048,"internally_reserved":0,"allocated":4026368,"data_stored":2241515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2440088256},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002016"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429898+0000","last_change":"2024-08-15T17:00:29.429898+0000","last_active":"2024-08-15T17:00:29.429898+0000","last_peered":"2024-08-15T17:00:29.429898+0000","last_clean":"2024-08-15T17:00:29.429898+0000","last_became_active":"2024-08-15T17:00:29.429650+0000","last_became_peered":"2024-08-15T17:00:29.429650+0000","last_unstale":"2024-08-15T17:00:29.429898+0000","last_undegraded":"2024-08-15T17:00:29.429898+0000","last_fullsized":"2024-08-15T17:00:29.429898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:50:24.059618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432521+0000","last_change":"2024-08-15T17:00:29.432521+0000","last_active":"2024-08-15T17:00:29.432521+0000","last_peered":"2024-08-15T17:00:29.432521+0000","last_clean":"2024-08-15T17:00:29.432521+0000","last_became_active":"2024-08-15T17:00:29.431901+0000","last_became_peered":"2024-08-15T17:00:29.431901+0000","last_unstale":"2024-08-15T17:00:29.432521+0000","last_undegraded":"2024-08-15T17:00:29.432521+0000","last_fullsized":"2024-08-15T17:00:29.432521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:48.751217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429949+0000","last_change":"2024-08-15T17:00:29.427098+0000","last_active":"2024-08-15T17:00:29.429949+0000","last_peered":"2024-08-15T17:00:29.429949+0000","last_clean":"2024-08-15T17:00:29.429949+0000","last_became_active":"2024-08-15T17:00:29.425645+0000","last_became_peered":"2024-08-15T17:00:29.425645+0000","last_unstale":"2024-08-15T17:00:29.429949+0000","last_undegraded":"2024-08-15T17:00:29.429949+0000","last_fullsized":"2024-08-15T17:00:29.429949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:30:40.296434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430965+0000","last_change":"2024-08-15T17:00:29.430965+0000","last_active":"2024-08-15T17:00:29.430965+0000","last_peered":"2024-08-15T17:00:29.430965+0000","last_clean":"2024-08-15T17:00:29.430965+0000","last_became_active":"2024-08-15T17:00:29.430705+0000","last_became_peered":"2024-08-15T17:00:29.430705+0000","last_unstale":"2024-08-15T17:00:29.430965+0000","last_undegraded":"2024-08-15T17:00:29.430965+0000","last_fullsized":"2024-08-15T17:00:29.430965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:46:21.695015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432193+0000","last_change":"2024-08-15T17:00:29.430770+0000","last_active":"2024-08-15T17:00:29.432193+0000","last_peered":"2024-08-15T17:00:29.432193+0000","last_clean":"2024-08-15T17:00:29.432193+0000","last_became_active":"2024-08-15T17:00:29.430216+0000","last_became_peered":"2024-08-15T17:00:29.430216+0000","last_unstale":"2024-08-15T17:00:29.432193+0000","last_undegraded":"2024-08-15T17:00:29.432193+0000","last_fullsized":"2024-08-15T17:00:29.432193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:10:09.167257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.428748+0000","last_change":"2024-08-15T17:00:29.428748+0000","last_active":"2024-08-15T17:00:29.428748+0000","last_peered":"2024-08-15T17:00:29.428748+0000","last_clean":"2024-08-15T17:00:29.428748+0000","last_became_active":"2024-08-15T17:00:29.428401+0000","last_became_peered":"2024-08-15T17:00:29.428401+0000","last_unstale":"2024-08-15T17:00:29.428748+0000","last_undegraded":"2024-08-15T17:00:29.428748+0000","last_fullsized":"2024-08-15T17:00:29.428748+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:07.974975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425759+0000","last_change":"2024-08-15T17:00:29.425759+0000","last_active":"2024-08-15T17:00:29.425759+0000","last_peered":"2024-08-15T17:00:29.425759+0000","last_clean":"2024-08-15T17:00:29.425759+0000","last_became_active":"2024-08-15T17:00:29.425171+0000","last_became_peered":"2024-08-15T17:00:29.425171+0000","last_unstale":"2024-08-15T17:00:29.425759+0000","last_undegraded":"2024-08-15T17:00:29.425759+0000","last_fullsized":"2024-08-15T17:00:29.425759+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:46:34.342266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423962+0000","last_change":"2024-08-15T17:00:29.423962+0000","last_active":"2024-08-15T17:00:29.423962+0000","last_peered":"2024-08-15T17:00:29.423962+0000","last_clean":"2024-08-15T17:00:29.423962+0000","last_became_active":"2024-08-15T17:00:29.422912+0000","last_became_peered":"2024-08-15T17:00:29.422912+0000","last_unstale":"2024-08-15T17:00:29.423962+0000","last_undegraded":"2024-08-15T17:00:29.423962+0000","last_fullsized":"2024-08-15T17:00:29.423962+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:39.997135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430950+0000","last_change":"2024-08-15T17:00:29.430013+0000","last_active":"2024-08-15T17:00:29.430950+0000","last_peered":"2024-08-15T17:00:29.430950+0000","last_clean":"2024-08-15T17:00:29.430950+0000","last_became_active":"2024-08-15T17:00:29.429695+0000","last_became_peered":"2024-08-15T17:00:29.429695+0000","last_unstale":"2024-08-15T17:00:29.430950+0000","last_undegraded":"2024-08-15T17:00:29.430950+0000","last_fullsized":"2024-08-15T17:00:29.430950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:16:50.162772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.433699+0000","last_change":"2024-08-15T17:00:29.433699+0000","last_active":"2024-08-15T17:00:29.433699+0000","last_peered":"2024-08-15T17:00:29.433699+0000","last_clean":"2024-08-15T17:00:29.433699+0000","last_became_active":"2024-08-15T17:00:29.433380+0000","last_became_peered":"2024-08-15T17:00:29.433380+0000","last_unstale":"2024-08-15T17:00:29.433699+0000","last_undegraded":"2024-08-15T17:00:29.433699+0000","last_fullsized":"2024-08-15T17:00:29.433699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:27.365495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425374+0000","last_change":"2024-08-15T17:00:29.425374+0000","last_active":"2024-08-15T17:00:29.425374+0000","last_peered":"2024-08-15T17:00:29.425374+0000","last_clean":"2024-08-15T17:00:29.425374+0000","last_became_active":"2024-08-15T17:00:29.425071+0000","last_became_peered":"2024-08-15T17:00:29.425071+0000","last_unstale":"2024-08-15T17:00:29.425374+0000","last_undegraded":"2024-08-15T17:00:29.425374+0000","last_fullsized":"2024-08-15T17:00:29.425374+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:38:05.927974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.436856+0000","last_change":"2024-08-15T17:00:29.433620+0000","last_active":"2024-08-15T17:00:29.436856+0000","last_peered":"2024-08-15T17:00:29.436856+0000","last_clean":"2024-08-15T17:00:29.436856+0000","last_became_active":"2024-08-15T17:00:29.432640+0000","last_became_peered":"2024-08-15T17:00:29.432640+0000","last_unstale":"2024-08-15T17:00:29.436856+0000","last_undegraded":"2024-08-15T17:00:29.436856+0000","last_fullsized":"2024-08-15T17:00:29.436856+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:58:38.549577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426860+0000","last_change":"2024-08-15T17:00:29.424724+0000","last_active":"2024-08-15T17:00:29.426860+0000","last_peered":"2024-08-15T17:00:29.426860+0000","last_clean":"2024-08-15T17:00:29.426860+0000","last_became_active":"2024-08-15T17:00:29.424290+0000","last_became_peered":"2024-08-15T17:00:29.424290+0000","last_unstale":"2024-08-15T17:00:29.426860+0000","last_undegraded":"2024-08-15T17:00:29.426860+0000","last_fullsized":"2024-08-15T17:00:29.426860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:15:24.707108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425845+0000","last_change":"2024-08-15T17:00:29.425845+0000","last_active":"2024-08-15T17:00:29.425845+0000","last_peered":"2024-08-15T17:00:29.425845+0000","last_clean":"2024-08-15T17:00:29.425845+0000","last_became_active":"2024-08-15T17:00:29.425540+0000","last_became_peered":"2024-08-15T17:00:29.425540+0000","last_unstale":"2024-08-15T17:00:29.425845+0000","last_undegraded":"2024-08-15T17:00:29.425845+0000","last_fullsized":"2024-08-15T17:00:29.425845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:39.997135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429880+0000","last_change":"2024-08-15T17:00:29.427429+0000","last_active":"2024-08-15T17:00:29.429880+0000","last_peered":"2024-08-15T17:00:29.429880+0000","last_clean":"2024-08-15T17:00:29.429880+0000","last_became_active":"2024-08-15T17:00:29.427145+0000","last_became_peered":"2024-08-15T17:00:29.427145+0000","last_unstale":"2024-08-15T17:00:29.429880+0000","last_undegraded":"2024-08-15T17:00:29.429880+0000","last_fullsized":"2024-08-15T17:00:29.429880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:39:49.776176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431597+0000","last_change":"2024-08-15T17:00:29.431597+0000","last_active":"2024-08-15T17:00:29.431597+0000","last_peered":"2024-08-15T17:00:29.431597+0000","last_clean":"2024-08-15T17:00:29.431597+0000","last_became_active":"2024-08-15T17:00:29.431337+0000","last_became_peered":"2024-08-15T17:00:29.431337+0000","last_unstale":"2024-08-15T17:00:29.431597+0000","last_undegraded":"2024-08-15T17:00:29.431597+0000","last_fullsized":"2024-08-15T17:00:29.431597+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:48.679569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426560+0000","last_change":"2024-08-15T17:00:29.426560+0000","last_active":"2024-08-15T17:00:29.426560+0000","last_peered":"2024-08-15T17:00:29.426560+0000","last_clean":"2024-08-15T17:00:29.426560+0000","last_became_active":"2024-08-15T17:00:29.426302+0000","last_became_peered":"2024-08-15T17:00:29.426302+0000","last_unstale":"2024-08-15T17:00:29.426560+0000","last_undegraded":"2024-08-15T17:00:29.426560+0000","last_fullsized":"2024-08-15T17:00:29.426560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:18:27.365495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430128+0000","last_change":"2024-08-15T17:00:29.430128+0000","last_active":"2024-08-15T17:00:29.430128+0000","last_peered":"2024-08-15T17:00:29.430128+0000","last_clean":"2024-08-15T17:00:29.430128+0000","last_became_active":"2024-08-15T17:00:29.429768+0000","last_became_peered":"2024-08-15T17:00:29.429768+0000","last_unstale":"2024-08-15T17:00:29.430128+0000","last_undegraded":"2024-08-15T17:00:29.430128+0000","last_fullsized":"2024-08-15T17:00:29.430128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:58.535553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432242+0000","last_change":"2024-08-15T17:00:29.432242+0000","last_active":"2024-08-15T17:00:29.432242+0000","last_peered":"2024-08-15T17:00:29.432242+0000","last_clean":"2024-08-15T17:00:29.432242+0000","last_became_active":"2024-08-15T17:00:29.431943+0000","last_became_peered":"2024-08-15T17:00:29.431943+0000","last_unstale":"2024-08-15T17:00:29.432242+0000","last_undegraded":"2024-08-15T17:00:29.432242+0000","last_fullsized":"2024-08-15T17:00:29.432242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:22:01.828223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423057+0000","last_change":"2024-08-15T17:00:29.421670+0000","last_active":"2024-08-15T17:00:29.423057+0000","last_peered":"2024-08-15T17:00:29.423057+0000","last_clean":"2024-08-15T17:00:29.423057+0000","last_became_active":"2024-08-15T17:00:29.421280+0000","last_became_peered":"2024-08-15T17:00:29.421280+0000","last_unstale":"2024-08-15T17:00:29.423057+0000","last_undegraded":"2024-08-15T17:00:29.423057+0000","last_fullsized":"2024-08-15T17:00:29.423057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:22:19.708581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430648+0000","last_change":"2024-08-15T17:00:29.427739+0000","last_active":"2024-08-15T17:00:29.430648+0000","last_peered":"2024-08-15T17:00:29.430648+0000","last_clean":"2024-08-15T17:00:29.430648+0000","last_became_active":"2024-08-15T17:00:29.427431+0000","last_became_peered":"2024-08-15T17:00:29.427431+0000","last_unstale":"2024-08-15T17:00:29.430648+0000","last_undegraded":"2024-08-15T17:00:29.430648+0000","last_fullsized":"2024-08-15T17:00:29.430648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:03:07.391680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437274+0000","last_change":"2024-08-15T17:00:29.432905+0000","last_active":"2024-08-15T17:00:29.437274+0000","last_peered":"2024-08-15T17:00:29.437274+0000","last_clean":"2024-08-15T17:00:29.437274+0000","last_became_active":"2024-08-15T17:00:29.432623+0000","last_became_peered":"2024-08-15T17:00:29.432623+0000","last_unstale":"2024-08-15T17:00:29.437274+0000","last_undegraded":"2024-08-15T17:00:29.437274+0000","last_fullsized":"2024-08-15T17:00:29.437274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:20:21.248696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422064+0000","last_change":"2024-08-15T17:00:29.422064+0000","last_active":"2024-08-15T17:00:29.422064+0000","last_peered":"2024-08-15T17:00:29.422064+0000","last_clean":"2024-08-15T17:00:29.422064+0000","last_became_active":"2024-08-15T17:00:29.421809+0000","last_became_peered":"2024-08-15T17:00:29.421809+0000","last_unstale":"2024-08-15T17:00:29.422064+0000","last_undegraded":"2024-08-15T17:00:29.422064+0000","last_fullsized":"2024-08-15T17:00:29.422064+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:28:54.704212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426569+0000","last_change":"2024-08-15T17:00:29.421506+0000","last_active":"2024-08-15T17:00:29.426569+0000","last_peered":"2024-08-15T17:00:29.426569+0000","last_clean":"2024-08-15T17:00:29.426569+0000","last_became_active":"2024-08-15T17:00:29.421155+0000","last_became_peered":"2024-08-15T17:00:29.421155+0000","last_unstale":"2024-08-15T17:00:29.426569+0000","last_undegraded":"2024-08-15T17:00:29.426569+0000","last_fullsized":"2024-08-15T17:00:29.426569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:22:13.180957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.424387+0000","last_change":"2024-08-15T17:00:29.424387+0000","last_active":"2024-08-15T17:00:29.424387+0000","last_peered":"2024-08-15T17:00:29.424387+0000","last_clean":"2024-08-15T17:00:29.424387+0000","last_became_active":"2024-08-15T17:00:29.424096+0000","last_became_peered":"2024-08-15T17:00:29.424096+0000","last_unstale":"2024-08-15T17:00:29.424387+0000","last_undegraded":"2024-08-15T17:00:29.424387+0000","last_fullsized":"2024-08-15T17:00:29.424387+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:36:10.787160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.432782+0000","last_change":"2024-08-15T17:00:29.432782+0000","last_active":"2024-08-15T17:00:29.432782+0000","last_peered":"2024-08-15T17:00:29.432782+0000","last_clean":"2024-08-15T17:00:29.432782+0000","last_became_active":"2024-08-15T17:00:29.432265+0000","last_became_peered":"2024-08-15T17:00:29.432265+0000","last_unstale":"2024-08-15T17:00:29.432782+0000","last_undegraded":"2024-08-15T17:00:29.432782+0000","last_fullsized":"2024-08-15T17:00:29.432782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425813+0000","last_change":"2024-08-15T17:00:29.425813+0000","last_active":"2024-08-15T17:00:29.425813+0000","last_peered":"2024-08-15T17:00:29.425813+0000","last_clean":"2024-08-15T17:00:29.425813+0000","last_became_active":"2024-08-15T17:00:29.425448+0000","last_became_peered":"2024-08-15T17:00:29.425448+0000","last_unstale":"2024-08-15T17:00:29.425813+0000","last_undegraded":"2024-08-15T17:00:29.425813+0000","last_fullsized":"2024-08-15T17:00:29.425813+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:28:22.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426700+0000","last_change":"2024-08-15T17:00:29.422214+0000","last_active":"2024-08-15T17:00:29.426700+0000","last_peered":"2024-08-15T17:00:29.426700+0000","last_clean":"2024-08-15T17:00:29.426700+0000","last_became_active":"2024-08-15T17:00:29.421956+0000","last_became_peered":"2024-08-15T17:00:29.421956+0000","last_unstale":"2024-08-15T17:00:29.426700+0000","last_undegraded":"2024-08-15T17:00:29.426700+0000","last_fullsized":"2024-08-15T17:00:29.426700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:25:21.682359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437544+0000","last_change":"2024-08-15T17:00:29.432980+0000","last_active":"2024-08-15T17:00:29.437544+0000","last_peered":"2024-08-15T17:00:29.437544+0000","last_clean":"2024-08-15T17:00:29.437544+0000","last_became_active":"2024-08-15T17:00:29.432677+0000","last_became_peered":"2024-08-15T17:00:29.432677+0000","last_unstale":"2024-08-15T17:00:29.437544+0000","last_undegraded":"2024-08-15T17:00:29.437544+0000","last_fullsized":"2024-08-15T17:00:29.437544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:41:17.776352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426574+0000","last_change":"2024-08-15T17:00:29.426574+0000","last_active":"2024-08-15T17:00:29.426574+0000","last_peered":"2024-08-15T17:00:29.426574+0000","last_clean":"2024-08-15T17:00:29.426574+0000","last_became_active":"2024-08-15T17:00:29.426351+0000","last_became_peered":"2024-08-15T17:00:29.426351+0000","last_unstale":"2024-08-15T17:00:29.426574+0000","last_undegraded":"2024-08-15T17:00:29.426574+0000","last_fullsized":"2024-08-15T17:00:29.426574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:45:05.108372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425695+0000","last_change":"2024-08-15T17:00:29.425695+0000","last_active":"2024-08-15T17:00:29.425695+0000","last_peered":"2024-08-15T17:00:29.425695+0000","last_clean":"2024-08-15T17:00:29.425695+0000","last_became_active":"2024-08-15T17:00:29.425222+0000","last_became_peered":"2024-08-15T17:00:29.425222+0000","last_unstale":"2024-08-15T17:00:29.425695+0000","last_undegraded":"2024-08-15T17:00:29.425695+0000","last_fullsized":"2024-08-15T17:00:29.425695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:27.346042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426770+0000","last_change":"2024-08-15T17:00:29.426770+0000","last_active":"2024-08-15T17:00:29.426770+0000","last_peered":"2024-08-15T17:00:29.426770+0000","last_clean":"2024-08-15T17:00:29.426770+0000","last_became_active":"2024-08-15T17:00:29.423369+0000","last_became_peered":"2024-08-15T17:00:29.423369+0000","last_unstale":"2024-08-15T17:00:29.426770+0000","last_undegraded":"2024-08-15T17:00:29.426770+0000","last_fullsized":"2024-08-15T17:00:29.426770+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:09:07.974975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423068+0000","last_change":"2024-08-15T17:00:29.423068+0000","last_active":"2024-08-15T17:00:29.423068+0000","last_peered":"2024-08-15T17:00:29.423068+0000","last_clean":"2024-08-15T17:00:29.423068+0000","last_became_active":"2024-08-15T17:00:29.422808+0000","last_became_peered":"2024-08-15T17:00:29.422808+0000","last_unstale":"2024-08-15T17:00:29.423068+0000","last_undegraded":"2024-08-15T17:00:29.423068+0000","last_fullsized":"2024-08-15T17:00:29.423068+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:05:37.411070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437762+0000","last_change":"2024-08-15T17:00:28.422039+0000","last_active":"2024-08-15T17:00:29.437762+0000","last_peered":"2024-08-15T17:00:29.437762+0000","last_clean":"2024-08-15T17:00:29.437762+0000","last_became_active":"2024-08-15T17:00:28.421736+0000","last_became_peered":"2024-08-15T17:00:28.421736+0000","last_unstale":"2024-08-15T17:00:29.437762+0000","last_undegraded":"2024-08-15T17:00:29.437762+0000","last_fullsized":"2024-08-15T17:00:29.437762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:00.360870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429335+0000","last_change":"2024-08-15T17:00:29.429335+0000","last_active":"2024-08-15T17:00:29.429335+0000","last_peered":"2024-08-15T17:00:29.429335+0000","last_clean":"2024-08-15T17:00:29.429335+0000","last_became_active":"2024-08-15T17:00:29.429007+0000","last_became_peered":"2024-08-15T17:00:29.429007+0000","last_unstale":"2024-08-15T17:00:29.429335+0000","last_undegraded":"2024-08-15T17:00:29.429335+0000","last_fullsized":"2024-08-15T17:00:29.429335+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:46:57.001279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416516+0000","last_change":"2024-08-15T17:00:28.425767+0000","last_active":"2024-08-15T17:00:29.416516+0000","last_peered":"2024-08-15T17:00:29.416516+0000","last_clean":"2024-08-15T17:00:29.416516+0000","last_became_active":"2024-08-15T17:00:28.425429+0000","last_became_peered":"2024-08-15T17:00:28.425429+0000","last_unstale":"2024-08-15T17:00:29.416516+0000","last_undegraded":"2024-08-15T17:00:29.416516+0000","last_fullsized":"2024-08-15T17:00:29.416516+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:35:45.386530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437208+0000","last_change":"2024-08-15T17:00:29.432466+0000","last_active":"2024-08-15T17:00:29.437208+0000","last_peered":"2024-08-15T17:00:29.437208+0000","last_clean":"2024-08-15T17:00:29.437208+0000","last_became_active":"2024-08-15T17:00:29.432149+0000","last_became_peered":"2024-08-15T17:00:29.432149+0000","last_unstale":"2024-08-15T17:00:29.437208+0000","last_undegraded":"2024-08-15T17:00:29.437208+0000","last_fullsized":"2024-08-15T17:00:29.437208+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:11:56.756218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.447267+0000","last_change":"2024-08-15T17:00:28.429121+0000","last_active":"2024-08-15T17:00:29.447267+0000","last_peered":"2024-08-15T17:00:29.447267+0000","last_clean":"2024-08-15T17:00:29.447267+0000","last_became_active":"2024-08-15T17:00:28.428806+0000","last_became_peered":"2024-08-15T17:00:28.428806+0000","last_unstale":"2024-08-15T17:00:29.447267+0000","last_undegraded":"2024-08-15T17:00:29.447267+0000","last_fullsized":"2024-08-15T17:00:29.447267+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:50:23.054325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423662+0000","last_change":"2024-08-15T17:00:29.423662+0000","last_active":"2024-08-15T17:00:29.423662+0000","last_peered":"2024-08-15T17:00:29.423662+0000","last_clean":"2024-08-15T17:00:29.423662+0000","last_became_active":"2024-08-15T17:00:29.423373+0000","last_became_peered":"2024-08-15T17:00:29.423373+0000","last_unstale":"2024-08-15T17:00:29.423662+0000","last_undegraded":"2024-08-15T17:00:29.423662+0000","last_fullsized":"2024-08-15T17:00:29.423662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:46:34.342266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.440878+0000","last_change":"2024-08-15T17:00:28.424224+0000","last_active":"2024-08-15T17:00:29.440878+0000","last_peered":"2024-08-15T17:00:29.440878+0000","last_clean":"2024-08-15T17:00:29.440878+0000","last_became_active":"2024-08-15T17:00:28.423938+0000","last_became_peered":"2024-08-15T17:00:28.423938+0000","last_unstale":"2024-08-15T17:00:29.440878+0000","last_undegraded":"2024-08-15T17:00:29.440878+0000","last_fullsized":"2024-08-15T17:00:29.440878+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:16:51.443419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.421401+0000","last_change":"2024-08-15T17:00:29.421401+0000","last_active":"2024-08-15T17:00:29.421401+0000","last_peered":"2024-08-15T17:00:29.421401+0000","last_clean":"2024-08-15T17:00:29.421401+0000","last_became_active":"2024-08-15T17:00:29.421005+0000","last_became_peered":"2024-08-15T17:00:29.421005+0000","last_unstale":"2024-08-15T17:00:29.421401+0000","last_undegraded":"2024-08-15T17:00:29.421401+0000","last_fullsized":"2024-08-15T17:00:29.421401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:18:17.891660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.442092+0000","last_change":"2024-08-15T17:00:28.423471+0000","last_active":"2024-08-15T17:00:29.442092+0000","last_peered":"2024-08-15T17:00:29.442092+0000","last_clean":"2024-08-15T17:00:29.442092+0000","last_became_active":"2024-08-15T17:00:28.422972+0000","last_became_peered":"2024-08-15T17:00:28.422972+0000","last_unstale":"2024-08-15T17:00:29.442092+0000","last_undegraded":"2024-08-15T17:00:29.442092+0000","last_fullsized":"2024-08-15T17:00:29.442092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:05:02.388046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429924+0000","last_change":"2024-08-15T17:00:29.427530+0000","last_active":"2024-08-15T17:00:29.429924+0000","last_peered":"2024-08-15T17:00:29.429924+0000","last_clean":"2024-08-15T17:00:29.429924+0000","last_became_active":"2024-08-15T17:00:29.427256+0000","last_became_peered":"2024-08-15T17:00:29.427256+0000","last_unstale":"2024-08-15T17:00:29.429924+0000","last_undegraded":"2024-08-15T17:00:29.429924+0000","last_fullsized":"2024-08-15T17:00:29.429924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:03:13.551921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.445851+0000","last_change":"2024-08-15T17:00:28.423304+0000","last_active":"2024-08-15T17:00:29.445851+0000","last_peered":"2024-08-15T17:00:29.445851+0000","last_clean":"2024-08-15T17:00:29.445851+0000","last_became_active":"2024-08-15T17:00:28.422986+0000","last_became_peered":"2024-08-15T17:00:28.422986+0000","last_unstale":"2024-08-15T17:00:29.445851+0000","last_undegraded":"2024-08-15T17:00:29.445851+0000","last_fullsized":"2024-08-15T17:00:29.445851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:59.476023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430188+0000","last_change":"2024-08-15T17:00:29.424259+0000","last_active":"2024-08-15T17:00:29.430188+0000","last_peered":"2024-08-15T17:00:29.430188+0000","last_clean":"2024-08-15T17:00:29.430188+0000","last_became_active":"2024-08-15T17:00:29.423818+0000","last_became_peered":"2024-08-15T17:00:29.423818+0000","last_unstale":"2024-08-15T17:00:29.430188+0000","last_undegraded":"2024-08-15T17:00:29.430188+0000","last_fullsized":"2024-08-15T17:00:29.430188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:28:54.704212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.426356+0000","last_change":"2024-08-15T17:00:28.422923+0000","last_active":"2024-08-15T17:00:28.426356+0000","last_peered":"2024-08-15T17:00:28.426356+0000","last_clean":"2024-08-15T17:00:28.426356+0000","last_became_active":"2024-08-15T17:00:28.422471+0000","last_became_peered":"2024-08-15T17:00:28.422471+0000","last_unstale":"2024-08-15T17:00:28.426356+0000","last_undegraded":"2024-08-15T17:00:28.426356+0000","last_fullsized":"2024-08-15T17:00:28.426356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:01.393488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416623+0000","last_change":"2024-08-15T17:00:29.416623+0000","last_active":"2024-08-15T17:00:29.416623+0000","last_peered":"2024-08-15T17:00:29.416623+0000","last_clean":"2024-08-15T17:00:29.416623+0000","last_became_active":"2024-08-15T17:00:29.415765+0000","last_became_peered":"2024-08-15T17:00:29.415765+0000","last_unstale":"2024-08-15T17:00:29.416623+0000","last_undegraded":"2024-08-15T17:00:29.416623+0000","last_fullsized":"2024-08-15T17:00:29.416623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:28:26.456498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.416767+0000","last_change":"2024-08-15T17:00:28.419871+0000","last_active":"2024-08-15T17:00:29.416767+0000","last_peered":"2024-08-15T17:00:29.416767+0000","last_clean":"2024-08-15T17:00:29.416767+0000","last_became_active":"2024-08-15T17:00:28.419568+0000","last_became_peered":"2024-08-15T17:00:28.419568+0000","last_unstale":"2024-08-15T17:00:29.416767+0000","last_undegraded":"2024-08-15T17:00:29.416767+0000","last_fullsized":"2024-08-15T17:00:29.416767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:54:20.081307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.436887+0000","last_change":"2024-08-15T17:00:29.433377+0000","last_active":"2024-08-15T17:00:29.436887+0000","last_peered":"2024-08-15T17:00:29.436887+0000","last_clean":"2024-08-15T17:00:29.436887+0000","last_became_active":"2024-08-15T17:00:29.433110+0000","last_became_peered":"2024-08-15T17:00:29.433110+0000","last_unstale":"2024-08-15T17:00:29.436887+0000","last_undegraded":"2024-08-15T17:00:29.436887+0000","last_fullsized":"2024-08-15T17:00:29.436887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:45:53.355016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429272+0000","last_change":"2024-08-15T17:00:28.427147+0000","last_active":"2024-08-15T17:00:28.429272+0000","last_peered":"2024-08-15T17:00:28.429272+0000","last_clean":"2024-08-15T17:00:28.429272+0000","last_became_active":"2024-08-15T17:00:28.426821+0000","last_became_peered":"2024-08-15T17:00:28.426821+0000","last_unstale":"2024-08-15T17:00:28.429272+0000","last_undegraded":"2024-08-15T17:00:28.429272+0000","last_fullsized":"2024-08-15T17:00:28.429272+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:19:19.327077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429298+0000","last_change":"2024-08-15T17:00:29.429298+0000","last_active":"2024-08-15T17:00:29.429298+0000","last_peered":"2024-08-15T17:00:29.429298+0000","last_clean":"2024-08-15T17:00:29.429298+0000","last_became_active":"2024-08-15T17:00:29.428959+0000","last_became_peered":"2024-08-15T17:00:29.428959+0000","last_unstale":"2024-08-15T17:00:29.429298+0000","last_undegraded":"2024-08-15T17:00:29.429298+0000","last_fullsized":"2024-08-15T17:00:29.429298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:06.118124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.424041+0000","last_change":"2024-08-15T17:00:28.423122+0000","last_active":"2024-08-15T17:00:28.424041+0000","last_peered":"2024-08-15T17:00:28.424041+0000","last_clean":"2024-08-15T17:00:28.424041+0000","last_became_active":"2024-08-15T17:00:28.422810+0000","last_became_peered":"2024-08-15T17:00:28.422810+0000","last_unstale":"2024-08-15T17:00:28.424041+0000","last_undegraded":"2024-08-15T17:00:28.424041+0000","last_fullsized":"2024-08-15T17:00:28.424041+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:43.420539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431800+0000","last_change":"2024-08-15T17:00:29.431800+0000","last_active":"2024-08-15T17:00:29.431800+0000","last_peered":"2024-08-15T17:00:29.431800+0000","last_clean":"2024-08-15T17:00:29.431800+0000","last_became_active":"2024-08-15T17:00:29.430099+0000","last_became_peered":"2024-08-15T17:00:29.430099+0000","last_unstale":"2024-08-15T17:00:29.431800+0000","last_undegraded":"2024-08-15T17:00:29.431800+0000","last_fullsized":"2024-08-15T17:00:29.431800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:41:48.679569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437603+0000","last_change":"2024-08-15T17:00:28.424682+0000","last_active":"2024-08-15T17:00:29.437603+0000","last_peered":"2024-08-15T17:00:29.437603+0000","last_clean":"2024-08-15T17:00:29.437603+0000","last_became_active":"2024-08-15T17:00:28.424235+0000","last_became_peered":"2024-08-15T17:00:28.424235+0000","last_unstale":"2024-08-15T17:00:29.437603+0000","last_undegraded":"2024-08-15T17:00:29.437603+0000","last_fullsized":"2024-08-15T17:00:29.437603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:04:49.362272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429932+0000","last_change":"2024-08-15T17:00:29.426875+0000","last_active":"2024-08-15T17:00:29.429932+0000","last_peered":"2024-08-15T17:00:29.429932+0000","last_clean":"2024-08-15T17:00:29.429932+0000","last_became_active":"2024-08-15T17:00:29.425983+0000","last_became_peered":"2024-08-15T17:00:29.425983+0000","last_unstale":"2024-08-15T17:00:29.429932+0000","last_undegraded":"2024-08-15T17:00:29.429932+0000","last_fullsized":"2024-08-15T17:00:29.429932+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:57:53.772660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.447874+0000","last_change":"2024-08-15T17:00:28.425753+0000","last_active":"2024-08-15T17:00:29.447874+0000","last_peered":"2024-08-15T17:00:29.447874+0000","last_clean":"2024-08-15T17:00:29.447874+0000","last_became_active":"2024-08-15T17:00:28.425178+0000","last_became_peered":"2024-08-15T17:00:28.425178+0000","last_unstale":"2024-08-15T17:00:29.447874+0000","last_undegraded":"2024-08-15T17:00:29.447874+0000","last_fullsized":"2024-08-15T17:00:29.447874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:46:55.995987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.426795+0000","last_change":"2024-08-15T17:00:29.426795+0000","last_active":"2024-08-15T17:00:29.426795+0000","last_peered":"2024-08-15T17:00:29.426795+0000","last_clean":"2024-08-15T17:00:29.426795+0000","last_became_active":"2024-08-15T17:00:29.426452+0000","last_became_peered":"2024-08-15T17:00:29.426452+0000","last_unstale":"2024-08-15T17:00:29.426795+0000","last_undegraded":"2024-08-15T17:00:29.426795+0000","last_fullsized":"2024-08-15T17:00:29.426795+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:38:05.927974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439253+0000","last_change":"2024-08-15T17:00:28.420058+0000","last_active":"2024-08-15T17:00:29.439253+0000","last_peered":"2024-08-15T17:00:29.439253+0000","last_clean":"2024-08-15T17:00:29.439253+0000","last_became_active":"2024-08-15T17:00:28.419386+0000","last_became_peered":"2024-08-15T17:00:28.419386+0000","last_unstale":"2024-08-15T17:00:29.439253+0000","last_undegraded":"2024-08-15T17:00:29.439253+0000","last_fullsized":"2024-08-15T17:00:29.439253+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:13:30.688015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.429274+0000","last_change":"2024-08-15T17:00:29.429274+0000","last_active":"2024-08-15T17:00:29.429274+0000","last_peered":"2024-08-15T17:00:29.429274+0000","last_clean":"2024-08-15T17:00:29.429274+0000","last_became_active":"2024-08-15T17:00:29.428695+0000","last_became_peered":"2024-08-15T17:00:29.428695+0000","last_unstale":"2024-08-15T17:00:29.429274+0000","last_undegraded":"2024-08-15T17:00:29.429274+0000","last_fullsized":"2024-08-15T17:00:29.429274+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:49:07.734249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.427807+0000","last_change":"2024-08-15T17:00:28.424742+0000","last_active":"2024-08-15T17:00:28.427807+0000","last_peered":"2024-08-15T17:00:28.427807+0000","last_clean":"2024-08-15T17:00:28.427807+0000","last_became_active":"2024-08-15T17:00:28.424410+0000","last_became_peered":"2024-08-15T17:00:28.424410+0000","last_unstale":"2024-08-15T17:00:28.427807+0000","last_undegraded":"2024-08-15T17:00:28.427807+0000","last_fullsized":"2024-08-15T17:00:28.427807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:36:19.262350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425018+0000","last_change":"2024-08-15T17:00:29.425018+0000","last_active":"2024-08-15T17:00:29.425018+0000","last_peered":"2024-08-15T17:00:29.425018+0000","last_clean":"2024-08-15T17:00:29.425018+0000","last_became_active":"2024-08-15T17:00:29.423702+0000","last_became_peered":"2024-08-15T17:00:29.423702+0000","last_unstale":"2024-08-15T17:00:29.425018+0000","last_undegraded":"2024-08-15T17:00:29.425018+0000","last_fullsized":"2024-08-15T17:00:29.425018+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T04:34:11.418135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.426376+0000","last_change":"2024-08-15T17:00:28.423019+0000","last_active":"2024-08-15T17:00:28.426376+0000","last_peered":"2024-08-15T17:00:28.426376+0000","last_clean":"2024-08-15T17:00:28.426376+0000","last_became_active":"2024-08-15T17:00:28.422097+0000","last_became_peered":"2024-08-15T17:00:28.422097+0000","last_unstale":"2024-08-15T17:00:28.426376+0000","last_undegraded":"2024-08-15T17:00:28.426376+0000","last_fullsized":"2024-08-15T17:00:28.426376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:20:25.523576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423544+0000","last_change":"2024-08-15T17:00:29.423544+0000","last_active":"2024-08-15T17:00:29.423544+0000","last_peered":"2024-08-15T17:00:29.423544+0000","last_clean":"2024-08-15T17:00:29.423544+0000","last_became_active":"2024-08-15T17:00:29.423275+0000","last_became_peered":"2024-08-15T17:00:29.423275+0000","last_unstale":"2024-08-15T17:00:29.423544+0000","last_undegraded":"2024-08-15T17:00:29.423544+0000","last_fullsized":"2024-08-15T17:00:29.423544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:12:02.398781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.448558+0000","last_change":"2024-08-15T17:00:28.428441+0000","last_active":"2024-08-15T17:00:29.448558+0000","last_peered":"2024-08-15T17:00:29.448558+0000","last_clean":"2024-08-15T17:00:29.448558+0000","last_became_active":"2024-08-15T17:00:28.428136+0000","last_became_peered":"2024-08-15T17:00:28.428136+0000","last_unstale":"2024-08-15T17:00:29.448558+0000","last_undegraded":"2024-08-15T17:00:29.448558+0000","last_fullsized":"2024-08-15T17:00:29.448558+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:20:26.340750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422610+0000","last_change":"2024-08-15T17:00:29.419470+0000","last_active":"2024-08-15T17:00:29.422610+0000","last_peered":"2024-08-15T17:00:29.422610+0000","last_clean":"2024-08-15T17:00:29.422610+0000","last_became_active":"2024-08-15T17:00:29.419187+0000","last_became_peered":"2024-08-15T17:00:29.419187+0000","last_unstale":"2024-08-15T17:00:29.422610+0000","last_undegraded":"2024-08-15T17:00:29.422610+0000","last_fullsized":"2024-08-15T17:00:29.422610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:39:49.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.419541+0000","last_change":"2024-08-15T17:00:28.423384+0000","last_active":"2024-08-15T17:00:29.419541+0000","last_peered":"2024-08-15T17:00:29.419541+0000","last_clean":"2024-08-15T17:00:29.419541+0000","last_became_active":"2024-08-15T17:00:28.423050+0000","last_became_peered":"2024-08-15T17:00:28.423050+0000","last_unstale":"2024-08-15T17:00:29.419541+0000","last_undegraded":"2024-08-15T17:00:29.419541+0000","last_fullsized":"2024-08-15T17:00:29.419541+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:00.360870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.428870+0000","last_change":"2024-08-15T17:00:29.424989+0000","last_active":"2024-08-15T17:00:29.428870+0000","last_peered":"2024-08-15T17:00:29.428870+0000","last_clean":"2024-08-15T17:00:29.428870+0000","last_became_active":"2024-08-15T17:00:29.424715+0000","last_became_peered":"2024-08-15T17:00:29.424715+0000","last_unstale":"2024-08-15T17:00:29.428870+0000","last_undegraded":"2024-08-15T17:00:29.428870+0000","last_fullsized":"2024-08-15T17:00:29.428870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:07:50.195961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439149+0000","last_change":"2024-08-15T17:00:28.418886+0000","last_active":"2024-08-15T17:00:29.439149+0000","last_peered":"2024-08-15T17:00:29.439149+0000","last_clean":"2024-08-15T17:00:29.439149+0000","last_became_active":"2024-08-15T17:00:28.418589+0000","last_became_peered":"2024-08-15T17:00:28.418589+0000","last_unstale":"2024-08-15T17:00:29.439149+0000","last_undegraded":"2024-08-15T17:00:29.439149+0000","last_fullsized":"2024-08-15T17:00:29.439149+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:09:43.420539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423612+0000","last_change":"2024-08-15T17:00:29.423612+0000","last_active":"2024-08-15T17:00:29.423612+0000","last_peered":"2024-08-15T17:00:29.423612+0000","last_clean":"2024-08-15T17:00:29.423612+0000","last_became_active":"2024-08-15T17:00:29.423257+0000","last_became_peered":"2024-08-15T17:00:29.423257+0000","last_unstale":"2024-08-15T17:00:29.423612+0000","last_undegraded":"2024-08-15T17:00:29.423612+0000","last_fullsized":"2024-08-15T17:00:29.423612+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:20:29.415382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.439226+0000","last_change":"2024-08-15T17:00:28.417723+0000","last_active":"2024-08-15T17:00:29.439226+0000","last_peered":"2024-08-15T17:00:29.439226+0000","last_clean":"2024-08-15T17:00:29.439226+0000","last_became_active":"2024-08-15T17:00:28.417299+0000","last_became_peered":"2024-08-15T17:00:28.417299+0000","last_unstale":"2024-08-15T17:00:29.439226+0000","last_undegraded":"2024-08-15T17:00:29.439226+0000","last_fullsized":"2024-08-15T17:00:29.439226+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:44:51.570213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.424585+0000","last_change":"2024-08-15T17:00:29.422313+0000","last_active":"2024-08-15T17:00:29.424585+0000","last_peered":"2024-08-15T17:00:29.424585+0000","last_clean":"2024-08-15T17:00:29.424585+0000","last_became_active":"2024-08-15T17:00:29.422019+0000","last_became_peered":"2024-08-15T17:00:29.422019+0000","last_unstale":"2024-08-15T17:00:29.424585+0000","last_undegraded":"2024-08-15T17:00:29.424585+0000","last_fullsized":"2024-08-15T17:00:29.424585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:17:46.929433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.412637+0000","last_change":"2024-08-15T17:00:28.423106+0000","last_active":"2024-08-15T17:00:29.412637+0000","last_peered":"2024-08-15T17:00:29.412637+0000","last_clean":"2024-08-15T17:00:29.412637+0000","last_became_active":"2024-08-15T17:00:28.422571+0000","last_became_peered":"2024-08-15T17:00:28.422571+0000","last_unstale":"2024-08-15T17:00:29.412637+0000","last_undegraded":"2024-08-15T17:00:29.412637+0000","last_fullsized":"2024-08-15T17:00:29.412637+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:51:15.739317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.423326+0000","last_change":"2024-08-15T17:00:29.422070+0000","last_active":"2024-08-15T17:00:29.423326+0000","last_peered":"2024-08-15T17:00:29.423326+0000","last_clean":"2024-08-15T17:00:29.423326+0000","last_became_active":"2024-08-15T17:00:29.421375+0000","last_became_peered":"2024-08-15T17:00:29.421375+0000","last_unstale":"2024-08-15T17:00:29.423326+0000","last_undegraded":"2024-08-15T17:00:29.423326+0000","last_fullsized":"2024-08-15T17:00:29.423326+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:52:38.910591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437006+0000","last_change":"2024-08-15T17:00:28.426143+0000","last_active":"2024-08-15T17:00:29.437006+0000","last_peered":"2024-08-15T17:00:29.437006+0000","last_clean":"2024-08-15T17:00:29.437006+0000","last_became_active":"2024-08-15T17:00:28.425808+0000","last_became_peered":"2024-08-15T17:00:28.425808+0000","last_unstale":"2024-08-15T17:00:29.437006+0000","last_undegraded":"2024-08-15T17:00:29.437006+0000","last_fullsized":"2024-08-15T17:00:29.437006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:44:51.570213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431433+0000","last_change":"2024-08-15T17:00:29.428735+0000","last_active":"2024-08-15T17:00:29.431433+0000","last_peered":"2024-08-15T17:00:29.431433+0000","last_clean":"2024-08-15T17:00:29.431433+0000","last_became_active":"2024-08-15T17:00:29.427893+0000","last_became_peered":"2024-08-15T17:00:29.427893+0000","last_unstale":"2024-08-15T17:00:29.431433+0000","last_undegraded":"2024-08-15T17:00:29.431433+0000","last_fullsized":"2024-08-15T17:00:29.431433+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:26.774890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429086+0000","last_change":"2024-08-15T17:00:28.425736+0000","last_active":"2024-08-15T17:00:28.429086+0000","last_peered":"2024-08-15T17:00:28.429086+0000","last_clean":"2024-08-15T17:00:28.429086+0000","last_became_active":"2024-08-15T17:00:28.425410+0000","last_became_peered":"2024-08-15T17:00:28.425410+0000","last_unstale":"2024-08-15T17:00:28.429086+0000","last_undegraded":"2024-08-15T17:00:28.429086+0000","last_fullsized":"2024-08-15T17:00:28.429086+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:25.769597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430469+0000","last_change":"2024-08-15T17:00:29.427385+0000","last_active":"2024-08-15T17:00:29.430469+0000","last_peered":"2024-08-15T17:00:29.430469+0000","last_clean":"2024-08-15T17:00:29.430469+0000","last_became_active":"2024-08-15T17:00:29.427067+0000","last_became_peered":"2024-08-15T17:00:29.427067+0000","last_unstale":"2024-08-15T17:00:29.430469+0000","last_undegraded":"2024-08-15T17:00:29.430469+0000","last_fullsized":"2024-08-15T17:00:29.430469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T23:01:01.366163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.411756+0000","last_change":"2024-08-15T17:00:28.423412+0000","last_active":"2024-08-15T17:00:29.411756+0000","last_peered":"2024-08-15T17:00:29.411756+0000","last_clean":"2024-08-15T17:00:29.411756+0000","last_became_active":"2024-08-15T17:00:28.423160+0000","last_became_peered":"2024-08-15T17:00:28.423160+0000","last_unstale":"2024-08-15T17:00:29.411756+0000","last_undegraded":"2024-08-15T17:00:29.411756+0000","last_fullsized":"2024-08-15T17:00:29.411756+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:45:14.327141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.425911+0000","last_change":"2024-08-15T17:00:29.425911+0000","last_active":"2024-08-15T17:00:29.425911+0000","last_peered":"2024-08-15T17:00:29.425911+0000","last_clean":"2024-08-15T17:00:29.425911+0000","last_became_active":"2024-08-15T17:00:29.425522+0000","last_became_peered":"2024-08-15T17:00:29.425522+0000","last_unstale":"2024-08-15T17:00:29.425911+0000","last_undegraded":"2024-08-15T17:00:29.425911+0000","last_fullsized":"2024-08-15T17:00:29.425911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:28:22.039647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.412891+0000","last_change":"2024-08-15T17:00:28.425065+0000","last_active":"2024-08-15T17:00:29.412891+0000","last_peered":"2024-08-15T17:00:29.412891+0000","last_clean":"2024-08-15T17:00:29.412891+0000","last_became_active":"2024-08-15T17:00:28.424314+0000","last_became_peered":"2024-08-15T17:00:28.424314+0000","last_unstale":"2024-08-15T17:00:29.412891+0000","last_undegraded":"2024-08-15T17:00:29.412891+0000","last_fullsized":"2024-08-15T17:00:29.412891+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:50:42.488167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422760+0000","last_change":"2024-08-15T17:00:29.419198+0000","last_active":"2024-08-15T17:00:29.422760+0000","last_peered":"2024-08-15T17:00:29.422760+0000","last_clean":"2024-08-15T17:00:29.422760+0000","last_became_active":"2024-08-15T17:00:29.418831+0000","last_became_peered":"2024-08-15T17:00:29.418831+0000","last_unstale":"2024-08-15T17:00:29.422760+0000","last_undegraded":"2024-08-15T17:00:29.422760+0000","last_fullsized":"2024-08-15T17:00:29.422760+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:51:18.474310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.440400+0000","last_change":"2024-08-15T17:00:28.424578+0000","last_active":"2024-08-15T17:00:29.440400+0000","last_peered":"2024-08-15T17:00:29.440400+0000","last_clean":"2024-08-15T17:00:29.440400+0000","last_became_active":"2024-08-15T17:00:28.424252+0000","last_became_peered":"2024-08-15T17:00:28.424252+0000","last_unstale":"2024-08-15T17:00:29.440400+0000","last_undegraded":"2024-08-15T17:00:29.440400+0000","last_fullsized":"2024-08-15T17:00:29.440400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T02:17:45.924140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.430294+0000","last_change":"2024-08-15T17:00:29.428679+0000","last_active":"2024-08-15T17:00:29.430294+0000","last_peered":"2024-08-15T17:00:29.430294+0000","last_clean":"2024-08-15T17:00:29.430294+0000","last_became_active":"2024-08-15T17:00:29.428394+0000","last_became_peered":"2024-08-15T17:00:29.428394+0000","last_unstale":"2024-08-15T17:00:29.430294+0000","last_undegraded":"2024-08-15T17:00:29.430294+0000","last_fullsized":"2024-08-15T17:00:29.430294+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T18:24:32.992039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.428506+0000","last_change":"2024-08-15T17:00:28.426296+0000","last_active":"2024-08-15T17:00:28.428506+0000","last_peered":"2024-08-15T17:00:28.428506+0000","last_clean":"2024-08-15T17:00:28.428506+0000","last_became_active":"2024-08-15T17:00:28.425999+0000","last_became_peered":"2024-08-15T17:00:28.425999+0000","last_unstale":"2024-08-15T17:00:28.428506+0000","last_undegraded":"2024-08-15T17:00:28.428506+0000","last_fullsized":"2024-08-15T17:00:28.428506+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:13:30.688015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.431228+0000","last_change":"2024-08-15T17:00:29.431228+0000","last_active":"2024-08-15T17:00:29.431228+0000","last_peered":"2024-08-15T17:00:29.431228+0000","last_clean":"2024-08-15T17:00:29.431228+0000","last_became_active":"2024-08-15T17:00:29.430931+0000","last_became_peered":"2024-08-15T17:00:29.430931+0000","last_unstale":"2024-08-15T17:00:29.431228+0000","last_undegraded":"2024-08-15T17:00:29.431228+0000","last_fullsized":"2024-08-15T17:00:29.431228+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:39:49.111099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.425124+0000","last_change":"2024-08-15T17:00:28.418354+0000","last_active":"2024-08-15T17:00:28.425124+0000","last_peered":"2024-08-15T17:00:28.425124+0000","last_clean":"2024-08-15T17:00:28.425124+0000","last_became_active":"2024-08-15T17:00:28.418012+0000","last_became_peered":"2024-08-15T17:00:28.418012+0000","last_unstale":"2024-08-15T17:00:28.425124+0000","last_undegraded":"2024-08-15T17:00:28.425124+0000","last_fullsized":"2024-08-15T17:00:28.425124+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:30.678372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.427791+0000","last_change":"2024-08-15T17:00:29.427791+0000","last_active":"2024-08-15T17:00:29.427791+0000","last_peered":"2024-08-15T17:00:29.427791+0000","last_clean":"2024-08-15T17:00:29.427791+0000","last_became_active":"2024-08-15T17:00:29.427521+0000","last_became_peered":"2024-08-15T17:00:29.427521+0000","last_unstale":"2024-08-15T17:00:29.427791+0000","last_undegraded":"2024-08-15T17:00:29.427791+0000","last_fullsized":"2024-08-15T17:00:29.427791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:20:29.415382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.424959+0000","last_change":"2024-08-15T17:00:28.424509+0000","last_active":"2024-08-15T17:00:28.424959+0000","last_peered":"2024-08-15T17:00:28.424959+0000","last_clean":"2024-08-15T17:00:28.424959+0000","last_became_active":"2024-08-15T17:00:28.423796+0000","last_became_peered":"2024-08-15T17:00:28.423796+0000","last_unstale":"2024-08-15T17:00:28.424959+0000","last_undegraded":"2024-08-15T17:00:28.424959+0000","last_fullsized":"2024-08-15T17:00:28.424959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:32:59.476023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.427684+0000","last_change":"2024-08-15T17:00:29.427684+0000","last_active":"2024-08-15T17:00:29.427684+0000","last_peered":"2024-08-15T17:00:29.427684+0000","last_clean":"2024-08-15T17:00:29.427684+0000","last_became_active":"2024-08-15T17:00:29.427060+0000","last_became_peered":"2024-08-15T17:00:29.427060+0000","last_unstale":"2024-08-15T17:00:29.427684+0000","last_undegraded":"2024-08-15T17:00:29.427684+0000","last_fullsized":"2024-08-15T17:00:29.427684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:59:20.756910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.427648+0000","last_change":"2024-08-15T17:00:28.422373+0000","last_active":"2024-08-15T17:00:28.427648+0000","last_peered":"2024-08-15T17:00:28.427648+0000","last_clean":"2024-08-15T17:00:28.427648+0000","last_became_active":"2024-08-15T17:00:28.420142+0000","last_became_peered":"2024-08-15T17:00:28.420142+0000","last_unstale":"2024-08-15T17:00:28.427648+0000","last_undegraded":"2024-08-15T17:00:28.427648+0000","last_fullsized":"2024-08-15T17:00:28.427648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T03:01:25.769597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.433127+0000","last_change":"2024-08-15T17:00:29.433127+0000","last_active":"2024-08-15T17:00:29.433127+0000","last_peered":"2024-08-15T17:00:29.433127+0000","last_clean":"2024-08-15T17:00:29.433127+0000","last_became_active":"2024-08-15T17:00:29.432696+0000","last_became_peered":"2024-08-15T17:00:29.432696+0000","last_unstale":"2024-08-15T17:00:29.433127+0000","last_undegraded":"2024-08-15T17:00:29.433127+0000","last_fullsized":"2024-08-15T17:00:29.433127+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T19:19:21.726365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.411869+0000","last_change":"2024-08-15T17:00:28.419753+0000","last_active":"2024-08-15T17:00:29.411869+0000","last_peered":"2024-08-15T17:00:29.411869+0000","last_clean":"2024-08-15T17:00:29.411869+0000","last_became_active":"2024-08-15T17:00:28.419472+0000","last_became_peered":"2024-08-15T17:00:28.419472+0000","last_unstale":"2024-08-15T17:00:29.411869+0000","last_undegraded":"2024-08-15T17:00:29.411869+0000","last_fullsized":"2024-08-15T17:00:29.411869+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T00:45:59.291885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.437305+0000","last_change":"2024-08-15T17:00:29.432941+0000","last_active":"2024-08-15T17:00:29.437305+0000","last_peered":"2024-08-15T17:00:29.437305+0000","last_clean":"2024-08-15T17:00:29.437305+0000","last_became_active":"2024-08-15T17:00:29.432650+0000","last_became_peered":"2024-08-15T17:00:29.432650+0000","last_unstale":"2024-08-15T17:00:29.437305+0000","last_undegraded":"2024-08-15T17:00:29.437305+0000","last_fullsized":"2024-08-15T17:00:29.437305+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T22:49:16.516093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.429039+0000","last_change":"2024-08-15T17:00:28.427297+0000","last_active":"2024-08-15T17:00:28.429039+0000","last_peered":"2024-08-15T17:00:28.429039+0000","last_clean":"2024-08-15T17:00:28.429039+0000","last_became_active":"2024-08-15T17:00:28.426965+0000","last_became_peered":"2024-08-15T17:00:28.426965+0000","last_unstale":"2024-08-15T17:00:28.429039+0000","last_undegraded":"2024-08-15T17:00:28.429039+0000","last_fullsized":"2024-08-15T17:00:28.429039+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-17T01:34:24.126020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-15T17:00:29.422540+0000","last_change":"2024-08-15T17:00:29.422540+0000","last_active":"2024-08-15T17:00:29.422540+0000","last_peered":"2024-08-15T17:00:29.422540+0000","last_clean":"2024-08-15T17:00:29.422540+0000","last_became_active":"2024-08-15T17:00:29.422228+0000","last_became_peered":"2024-08-15T17:00:29.422228+0000","last_unstale":"2024-08-15T17:00:29.422540+0000","last_undegraded":"2024-08-15T17:00:29.422540+0000","last_fullsized":"2024-08-15T17:00:29.422540+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:28.390995+0000","last_clean_scrub_stamp":"2024-08-15T17:00:28.390995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:42:00.898538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.423804+0000","last_change":"2024-08-15T17:00:28.420779+0000","last_active":"2024-08-15T17:00:28.423804+0000","last_peered":"2024-08-15T17:00:28.423804+0000","last_clean":"2024-08-15T17:00:28.423804+0000","last_became_active":"2024-08-15T17:00:28.420422+0000","last_became_peered":"2024-08-15T17:00:28.420422+0000","last_unstale":"2024-08-15T17:00:28.423804+0000","last_undegraded":"2024-08-15T17:00:28.423804+0000","last_fullsized":"2024-08-15T17:00:28.423804+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T17:00:27.385702+0000","last_clean_scrub_stamp":"2024-08-15T17:00:27.385702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T21:01:30.678372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":169,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-08-15T17:00:28.416883+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:00:28.416883+0000","last_peered":"2024-08-15T17:00:28.416883+0000","last_clean":"2024-08-15T17:00:28.416883+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:00:28.416883+0000","last_undegraded":"2024-08-15T17:00:28.416883+0000","last_fullsized":"2024-08-15T17:00:28.416883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298276,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073564,"statfs":{"total":96636764160,"available":96331329536,"internally_reserved":0,"allocated":942080,"data_stored":721019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297704,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074136,"statfs":{"total":96636764160,"available":96331915264,"internally_reserved":0,"allocated":352256,"data_stored":131165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":364544,"data_stored":137509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]}]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":352256,"data_stored":130665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":360448,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542159,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074124,"statfs":{"total":96636764160,"available":96331902976,"internally_reserved":0,"allocated":360448,"data_stored":136919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":360448,"data_stored":132135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]}]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":933888,"data_stored":719968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:00:33.807 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-15T17:00:33.808 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-08-15T17:00:33.927 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 Processor -- start 2024-08-15T17:00:33.928 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 -- start start 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d801534b0 0x7f2d800cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d800b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d800ce7f0 con 0x7f2d800b2000 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d800b2480 con 0x7f2d801534b0 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800cea90 con 0x7f2d80153880 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d801534b0 0x7f2d800cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2d801534b0 0x7f2d800cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50960/0 (socket says 172.21.15.146:50960) 2024-08-15T17:00:33.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 -- 172.21.15.146:0/1605066025 learned_addr learned my addr 172.21.15.146:0/1605066025 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:33.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7e7fc700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d800b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 -- 172.21.15.146:0/1605066025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 msgr2=0x7f2d800b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d800b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 -- 172.21.15.146:0/1605066025 >> v1:172.21.15.161:6789/0 conn(0x7f2d800b2000 legacy=0x7f2d800d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 -- 172.21.15.146:0/1605066025 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d800ced70 con 0x7f2d801534b0 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7effd700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d800cf570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099b0 tx=0x7f2d70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4876fa4454767495 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7dffb700 1 -- 172.21.15.146:0/1605066025 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70009ea0 con 0x7f2d801534b0 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7dffb700 1 -- 172.21.15.146:0/1605066025 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000b860 con 0x7f2d801534b0 2024-08-15T17:00:33.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.917+0000 7f2d7dffb700 1 -- 172.21.15.146:0/1605066025 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000bc00 con 0x7f2d801534b0 2024-08-15T17:00:33.932 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- 172.21.15.146:0/1605066025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 msgr2=0x7f2d800cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d800cf570 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099b0 tx=0x7f2d70009250 comp rx=0 tx=0).stop 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- 172.21.15.146:0/1605066025 shutdown_connections 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d800b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/1605066025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d800cf570 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- 172.21.15.146:0/1605066025 >> 172.21.15.146:0/1605066025 conn(0x7f2d80066e00 msgr2=0x7f2d80067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- 172.21.15.146:0/1605066025 shutdown_connections 2024-08-15T17:00:33.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- 172.21.15.146:0/1605066025 wait complete. 2024-08-15T17:00:33.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 Processor -- start 2024-08-15T17:00:33.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.921+0000 7f2d7ffff700 1 -- start start 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 0x7f2d80152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d80138980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801501b0 con 0x7f2d801534b0 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8014dad0 con 0x7f2d800b2000 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8014df60 con 0x7f2d80153880 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50972/0 (socket says 172.21.15.146:50972) 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 -- 172.21.15.146:0/3598558471 learned_addr learned my addr 172.21.15.146:0/3598558471 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:33.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7effd700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 0x7f2d80152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7f7fe700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d80138980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 -- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 msgr2=0x7f2d80138980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d80138980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 -- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 msgr2=0x7f2d80152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 0x7f2d80152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 -- 172.21.15.146:0/3598558471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094f0 con 0x7f2d801534b0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7effd700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 0x7f2d80152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7f7fe700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d80138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7e7fc700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400e930 tx=0x7f2d740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400eed0 con 0x7f2d801534b0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7400c850 con 0x7f2d801534b0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8014cef0 con 0x7f2d801534b0 2024-08-15T17:00:33.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400cbf0 con 0x7f2d801534b0 2024-08-15T17:00:33.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.925+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8014d480 con 0x7f2d801534b0 2024-08-15T17:00:33.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8014dc60 con 0x7f2d801534b0 2024-08-15T17:00:33.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2d74013470 con 0x7f2d801534b0 2024-08-15T17:00:33.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d66ffd700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 0x7f2d6c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:33.940 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d7407ce00 con 0x7f2d801534b0 2024-08-15T17:00:33.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d7effd700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 0x7f2d6c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:33.941 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.929+0000 7f2d7effd700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 0x7f2d6c062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099b0 tx=0x7f2d70017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:33.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:33.937+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2d7404d250 con 0x7f2d801534b0 2024-08-15T17:00:34.162 INFO:tasks.ceph.mds.d.smithi161.stderr:2024-08-15T17:00:34.149+0000 7f48258d2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:00:34.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.153+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2d80070d10 con 0x7f2d801534b0 2024-08-15T17:00:34.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.161+0000 7f2d66ffd700 1 -- 172.21.15.146:0/3598558471 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2d74050870 con 0x7f2d801534b0 2024-08-15T17:00:34.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 msgr2=0x7f2d6c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.177 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 0x7f2d6c062d10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099b0 tx=0x7f2d70017000 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 msgr2=0x7f2d8014fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400e930 tx=0x7f2d740091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 shutdown_connections 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2d6c060860 0x7f2d6c062d10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d80153880 0x7f2d80138980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d801534b0 0x7f2d8014fc70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 --2- 172.21.15.146:0/3598558471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d800b2000 0x7f2d80152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 >> 172.21.15.146:0/3598558471 conn(0x7f2d80066e00 msgr2=0x7f2d800682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 shutdown_connections 2024-08-15T17:00:34.178 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.165+0000 7f2d7ffff700 1 -- 172.21.15.146:0/3598558471 wait complete. 2024-08-15T17:00:34.179 INFO:teuthology.orchestra.run.smithi146.stderr:updated caps for client.0 2024-08-15T17:00:34.193 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:00:34.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 Processor -- start 2024-08-15T17:00:34.298 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 -- start start 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa4f015c620 0x7fa4f00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f00c86e0 con 0x7fa4f00ad1b0 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f015d210 con 0x7fa4f015cd90 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.285+0000 7fa4f6236700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00cb850 con 0x7fa4f015c620 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5a35700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa4f00ad1b0 0x7fa4f00c8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45440/0 (socket says 172.21.15.146:45440) 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5a35700 1 -- 172.21.15.146:0/1594768309 learned_addr learned my addr 172.21.15.146:0/1594768309 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5234700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa4f015c620 0x7fa4f00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309876760 0 0) 0x7fa4f00c86e0 con 0x7fa4f00ad1b0 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4dc003570 con 0x7fa4f00ad1b0 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263364578 0 0) 0x7fa4f015d210 con 0x7fa4f015cd90 2024-08-15T17:00:34.300 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f00c86e0 con 0x7fa4f015cd90 2024-08-15T17:00:34.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5234700 1 -- 172.21.15.146:0/1594768309 >> v1:172.21.15.161:6789/0 conn(0x7fa4f015cd90 legacy=0x7fa4f00acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5234700 1 -- 172.21.15.146:0/1594768309 >> v1:172.21.15.146:6789/0 conn(0x7fa4f00ad1b0 legacy=0x7fa4f00c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5234700 1 -- 172.21.15.146:0/1594768309 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f015d490 con 0x7fa4f015c620 2024-08-15T17:00:34.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f5234700 1 --2- 172.21.15.146:0/1594768309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00b0320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0098e0 tx=0x7fa4ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578c8076345ae49 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:34.301 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec011630 con 0x7fa4f015c620 2024-08-15T17:00:34.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec011790 con 0x7fa4f015c620 2024-08-15T17:00:34.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4e3fff700 1 -- 172.21.15.146:0/1594768309 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec011b30 con 0x7fa4f015c620 2024-08-15T17:00:34.302 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 -- 172.21.15.146:0/1594768309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 msgr2=0x7fa4f00b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 --2- 172.21.15.146:0/1594768309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00b0320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0098e0 tx=0x7fa4ec0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 -- 172.21.15.146:0/1594768309 shutdown_connections 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 --2- 172.21.15.146:0/1594768309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00b0320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 -- 172.21.15.146:0/1594768309 >> 172.21.15.146:0/1594768309 conn(0x7fa4f00ca900 msgr2=0x7fa4f00cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 -- 172.21.15.146:0/1594768309 shutdown_connections 2024-08-15T17:00:34.303 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.289+0000 7fa4f6236700 1 -- 172.21.15.146:0/1594768309 wait complete. 2024-08-15T17:00:34.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.293+0000 7fa4f6236700 1 Processor -- start 2024-08-15T17:00:34.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.293+0000 7fa4f6236700 1 -- start start 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4f00ad1b0 0x7fa4f014a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4f015cd90 0x7fa4f00ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00aaee0 con 0x7fa4f015c620 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00aa8d0 con 0x7fa4f015cd90 2024-08-15T17:00:34.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00c5e00 con 0x7fa4f00ad1b0 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:50996/0 (socket says 172.21.15.146:50996) 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 -- 172.21.15.146:0/890650329 learned_addr learned my addr 172.21.15.146:0/890650329 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f5234700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4f00ad1b0 0x7fa4f014a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f5a35700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4f015cd90 0x7fa4f00ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 -- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4f00ad1b0 msgr2=0x7fa4f014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4f00ad1b0 0x7fa4f014a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 -- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4f015cd90 msgr2=0x7fa4f00ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4f015cd90 0x7fa4f00ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 -- 172.21.15.146:0/890650329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec009420 con 0x7fa4f015c620 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f4a33700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c580 tx=0x7fa4e4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4010510 con 0x7fa4f015c620 2024-08-15T17:00:34.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4010670 con 0x7fa4f015c620 2024-08-15T17:00:34.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f00aab70 con 0x7fa4f015c620 2024-08-15T17:00:34.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4010a30 con 0x7fa4f015c620 2024-08-15T17:00:34.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f014ad70 con 0x7fa4f015c620 2024-08-15T17:00:34.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f00ab050 con 0x7fa4f015c620 2024-08-15T17:00:34.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.297+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa4e401b400 con 0x7fa4f015c620 2024-08-15T17:00:34.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.301+0000 7fa4e17fa700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:34.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.301+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa4e407c8b0 con 0x7fa4f015c620 2024-08-15T17:00:34.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.301+0000 7fa4f5234700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:34.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.301+0000 7fa4f5234700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 0x7fa4d4062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0093f0 tx=0x7fa4ec017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:34.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.304+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa4e404cc60 con 0x7fa4f015c620 2024-08-15T17:00:34.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.500+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4f00aad00 con 0x7fa4f015c620 2024-08-15T17:00:34.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.500+0000 7fa4e17fa700 1 -- 172.21.15.146:0/890650329 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fa4e4050280 con 0x7fa4f015c620 2024-08-15T17:00:34.514 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:34.514 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4926,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 msgr2=0x7fa4d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 0x7fa4d4062d10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0093f0 tx=0x7fa4ec017000 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 msgr2=0x7fa4f00ae250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c580 tx=0x7fa4e4007e10 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 shutdown_connections 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa4d4060860 0x7fa4d4062d10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4f015cd90 0x7fa4f00ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa4f015c620 0x7fa4f00ae250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 --2- 172.21.15.146:0/890650329 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4f00ad1b0 0x7fa4f014a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:34.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 >> 172.21.15.146:0/890650329 conn(0x7fa4f00ca900 msgr2=0x7fa4f00c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:34.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 shutdown_connections 2024-08-15T17:00:34.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:34.504+0000 7fa4f6236700 1 -- 172.21.15.146:0/890650329 wait complete. 2024-08-15T17:00:34.518 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 14 2024-08-15T17:00:34.930 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-15T17:00:28.770829+0000', 'modified': '2024-08-15T17:00:32.438138+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4936}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4936': {'gid': 4936, 'name': 'd', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/2559773181', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2559773181}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2559773181}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-15T17:00:34.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-15T17:00:34.931 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-08-15T17:00:35.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 Processor -- start 2024-08-15T17:00:35.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 -- start start 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ed8155760 0x7f8ed80abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed806f250 0x7f8ed80c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80c87d0 con 0x7f8ed8155b30 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed80ac7d0 con 0x7f8ed8155760 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edf980700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80acab0 con 0x7f8ed806f250 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ed8155760 0x7f8ed80abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f8ed8155760 0x7f8ed80abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51010/0 (socket says 172.21.15.146:51010) 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 -- 172.21.15.146:0/390809236 learned_addr learned my addr 172.21.15.146:0/390809236 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed806f250 0x7f8ed80c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 -- 172.21.15.146:0/390809236 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed806f250 msgr2=0x7f8ed80c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed806f250 0x7f8ed80c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 -- 172.21.15.146:0/390809236 >> v1:172.21.15.161:6789/0 conn(0x7f8ed8155b30 legacy=0x7f8ed806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.036+0000 7f8edd71c700 1 -- 172.21.15.146:0/390809236 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80acd90 con 0x7f8ed8155760 2024-08-15T17:00:35.052 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edd71c700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155760 0x7f8ed80abf80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d1a0 tx=0x7f8ed400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9fe52cc48112a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8ecffff700 1 -- 172.21.15.146:0/390809236 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4010070 con 0x7f8ed8155760 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8ecffff700 1 -- 172.21.15.146:0/390809236 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400dc00 con 0x7f8ed8155760 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8ecffff700 1 -- 172.21.15.146:0/390809236 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4016400 con 0x7f8ed8155760 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- 172.21.15.146:0/390809236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155760 msgr2=0x7f8ed80abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155760 0x7f8ed80abf80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d1a0 tx=0x7f8ed400ba50 comp rx=0 tx=0).stop 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- 172.21.15.146:0/390809236 shutdown_connections 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed806f250 0x7f8ed80c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 --2- 172.21.15.146:0/390809236 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155760 0x7f8ed80abf80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- 172.21.15.146:0/390809236 >> 172.21.15.146:0/390809236 conn(0x7f8ed8067810 msgr2=0x7f8ed8067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- 172.21.15.146:0/390809236 shutdown_connections 2024-08-15T17:00:35.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- 172.21.15.146:0/390809236 wait complete. 2024-08-15T17:00:35.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 Processor -- start 2024-08-15T17:00:35.055 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.040+0000 7f8edf980700 1 -- start start 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed806f250 0x7f8ed8099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 0x7f8ed80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed8155fe0 con 0x7f8ed8155b30 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81562c0 con 0x7f8ed806f250 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80b0ee0 con 0x7f8ed8155760 2024-08-15T17:00:35.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51024/0 (socket says 172.21.15.146:51024) 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 -- 172.21.15.146:0/324147305 learned_addr learned my addr 172.21.15.146:0/324147305 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edd71c700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed806f250 0x7f8ed8099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 0x7f8ed80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 -- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 msgr2=0x7f8ed80b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 0x7f8ed80b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 -- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed806f250 msgr2=0x7f8ed8099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed806f250 0x7f8ed8099540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 -- 172.21.15.146:0/324147305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed400bcf0 con 0x7f8ed8155b30 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 0x7f8ed80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:35.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000ea80 tx=0x7f8ed00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8ecd7fa700 1 -- 172.21.15.146:0/324147305 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000c7f0 con 0x7f8ed8155b30 2024-08-15T17:00:35.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8ecd7fa700 1 -- 172.21.15.146:0/324147305 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000c970 con 0x7f8ed8155b30 2024-08-15T17:00:35.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8ecd7fa700 1 -- 172.21.15.146:0/324147305 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000cd30 con 0x7f8ed8155b30 2024-08-15T17:00:35.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed80b11c0 con 0x7f8ed8155b30 2024-08-15T17:00:35.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.044+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed80b0bf0 con 0x7f8ed8155b30 2024-08-15T17:00:35.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.048+0000 7f8ecd7fa700 1 -- 172.21.15.146:0/324147305 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8ed0013680 con 0x7f8ed8155b30 2024-08-15T17:00:35.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.048+0000 7f8ecd7fa700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 0x7f8ebc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.048+0000 7f8ecd7fa700 1 -- 172.21.15.146:0/324147305 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ed007ce50 con 0x7f8ed8155b30 2024-08-15T17:00:35.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.048+0000 7f8edd71c700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 0x7f8ebc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.048+0000 7f8edd71c700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 0x7f8ebc062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400ba90 tx=0x7f8ed401c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 Processor -- start 2024-08-15T17:00:35.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 -- start start 2024-08-15T17:00:35.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1dc8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 0x1dcbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d40ba0 con 0x1b94ae0 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d3cb40 con 0x1bdf790 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d020a0 con 0x1b94710 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51032/0 (socket says 172.21.15.146:51032) 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 -- 172.21.15.146:0/3321592508 learned_addr learned my addr 172.21.15.146:0/3321592508 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1dc8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 0x1dcbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 -- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 msgr2=0x1dc8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1dc8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 -- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 msgr2=0x1dcbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 0x1dcbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 -- 172.21.15.146:0/3321592508 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1be09e0 con 0x1b94ae0 2024-08-15T17:00:35.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 0x1dcbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:35.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d840 tx=0x7f8ed4023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3535f945002916d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ec27fc700 1 -- 172.21.15.146:0/3321592508 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4016cd0 con 0x1b94ae0 2024-08-15T17:00:35.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ec27fc700 1 -- 172.21.15.146:0/3321592508 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4016e50 con 0x1b94ae0 2024-08-15T17:00:35.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.052+0000 7f8ec27fc700 1 -- 172.21.15.146:0/3321592508 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed402a6b0 con 0x1b94ae0 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3321592508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 msgr2=0x1dc81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400d840 tx=0x7f8ed4023040 comp rx=0 tx=0).stop 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3321592508 shutdown_connections 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1bdf790 0x1dcbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1dc8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3321592508 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94ae0 0x1dc81d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3321592508 >> 172.21.15.146:0/3321592508 conn(0x1b94eb0 msgr2=0x1dce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3321592508 shutdown_connections 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3321592508 wait complete. 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- start start 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 Processor -- start 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1e15890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e15dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d01d40 con 0x1bdf790 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1cda970 con 0x1b94ae0 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d3c350 con 0x1b94710 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 10 client.0 _ll_register_callbacks cb 0x1cdef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8edd71c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1e15890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51034/0 (socket says 172.21.15.146:51034) 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 -- 172.21.15.146:0/2401452211 learned_addr learned my addr 172.21.15.146:0/2401452211 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e15dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 -- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 msgr2=0x1e15890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1e15890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 -- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 msgr2=0x1e15dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e15dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 -- 172.21.15.146:0/2401452211 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed80acd90 con 0x1bdf790 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c430 tx=0x7f8ed0056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67479fd2017e5ae3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0050aa0 con 0x1bdf790 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ed004f300 con 0x1bdf790 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.056+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ba2260 con 0x1bdf790 2024-08-15T17:00:35.074 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed0051410 con 0x1bdf790 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 10 client.5097 fetch_fsmap learned FSMap version 14 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x1d0a370 con 0x1bdf790 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ba2260 con 0x1bdf790 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 14) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8ed006b0c0 con 0x1bdf790 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 10 client.5097 fetch_fsmap finished waiting for FSMap version 14 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 10 client.5097 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 20 client.5097 populate_metadata read hostname 'smithi146' 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e1ea20 0x1e20e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 4 client.5097 mds_command: new command op to 4936 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 --> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1bec4e0 con 0x1e1ea20 2024-08-15T17:00:35.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8edd71c700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e1ea20 0x1e20e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 10 client.5097 ms_handle_connect on v2:172.21.15.161:6836/2559773181 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8edd71c700 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e1ea20 0x1e20e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13bb02b1666ff1b3 server_cookie=1a1ce2f2d47c4fae in_seq=0 out_seq=0 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 1 -- 172.21.15.146:0/2401452211 <== mds.0 v2:172.21.15.161:6836/2559773181 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x1bec4e0 con 0x1e1ea20 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 10 client.5097 handle_command_reply: tid=0 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 client.5097 shutdown 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 msgr2=0x1dd0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c430 tx=0x7f8ed0056040 comp rx=0 tx=0).stop 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 shutdown_connections 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1dd0430 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e1ea20 0x1e20e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e15dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/2401452211 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94710 0x1e15890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8eb77fe700 0 client.5097 ms_handle_reset on v2:172.21.15.161:6836/2559773181 2024-08-15T17:00:35.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 >> 172.21.15.146:0/2401452211 conn(0x1b94eb0 msgr2=0x1dc9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 shutdown_connections 2024-08-15T17:00:35.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.060+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/2401452211 wait complete. 2024-08-15T17:00:35.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.064+0000 7f8ee1c2f740 20 client.5097 trim_cache size 0 max 16384 2024-08-15T17:00:35.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.080+0000 7f8ee1c2f740 1 Processor -- start 2024-08-15T17:00:35.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.080+0000 7f8ee1c2f740 1 -- start start 2024-08-15T17:00:35.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94710 0x1dcf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94ae0 0x1dcf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d38650 con 0x1bdf790 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d3b9d0 con 0x1b94710 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1d42da0 con 0x1b94ae0 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51038/0 (socket says 172.21.15.146:51038) 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 -- 172.21.15.146:0/776989544 learned_addr learned my addr 172.21.15.146:0/776989544 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94ae0 0x1dcf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8edd71c700 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94710 0x1dcf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 -- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94ae0 msgr2=0x1dcf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94ae0 0x1dcf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 -- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94710 msgr2=0x1dcf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94710 0x1dcf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 -- 172.21.15.146:0/776989544 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1cf0d80 con 0x1bdf790 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ed004f2b0 tx=0x7f8ed0063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a805d975920c43 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ec2ffd700 1 -- 172.21.15.146:0/776989544 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed006b220 con 0x1bdf790 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ec2ffd700 1 -- 172.21.15.146:0/776989544 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000e040 con 0x1bdf790 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ec2ffd700 1 -- 172.21.15.146:0/776989544 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0050aa0 con 0x1bdf790 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/776989544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 msgr2=0x1e1edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ed004f2b0 tx=0x7f8ed0063000 comp rx=0 tx=0).stop 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/776989544 shutdown_connections 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1bdf790 0x1e1edf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1b94ae0 0x1dcf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/776989544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94710 0x1dcf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/776989544 >> 172.21.15.146:0/776989544 conn(0x1b94eb0 msgr2=0x1dcce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/776989544 shutdown_connections 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/776989544 wait complete. 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.084+0000 7f8ee1c2f740 1 -- start start 2024-08-15T17:00:35.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 Processor -- start 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e29c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1bdf790 0x1e2a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d3f200 con 0x1b94710 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x1d62910 con 0x1b94ae0 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x1cded30 con 0x1bdf790 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 10 client.0 _ll_register_callbacks cb 0x1b609d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51054/0 (socket says 172.21.15.146:51054) 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 -- 172.21.15.146:0/3155675647 learned_addr learned my addr 172.21.15.146:0/3155675647 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eddf1d700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1bdf790 0x1e2a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e29c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 -- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1bdf790 msgr2=0x1e2a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1bdf790 0x1e2a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 -- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 msgr2=0x1e29c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e29c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 -- 172.21.15.146:0/3155675647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1d0a370 con 0x1b94710 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edd71c700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4032000 tx=0x7f8ed401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cd84aeac5165d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4016520 con 0x1b94710 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ed4016e50 con 0x1b94710 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mon.0 v2:172.21.15.146:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8ed402a460 con 0x1b94710 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ed402a620 con 0x1b94710 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 10 client.5103 fetch_fsmap learned FSMap version 14 2024-08-15T17:00:35.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x1ce5130 con 0x1b94710 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mon.0 v2:172.21.15.146:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4016e50 con 0x1b94710 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mon.0 v2:172.21.15.146:3300/0 5 ==== fsmap(e 14) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8ed400a250 con 0x1b94710 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 10 client.5103 fetch_fsmap finished waiting for FSMap version 14 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 10 client.5103 resolve_mds: resolved name 'd' to daemon mds.d 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 20 client.5103 populate_metadata read hostname 'smithi146' 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e2a700 0x1e6d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 4 client.5103 mds_command: new command op to 4936 tid=0[{"prefix": "status"}] 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 --> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1c11260 con 0x1e2a700 2024-08-15T17:00:35.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.088+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e2a700 0x1e6d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edcf1b700 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e2a700 0x1e6d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb4d5021545daf36 server_cookie=6ac39d2a75f9e38b in_seq=0 out_seq=0 2024-08-15T17:00:35.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8eb77fe700 10 client.5103 ms_handle_connect on v2:172.21.15.161:6836/2559773181 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8eb77fe700 1 -- 172.21.15.146:0/3155675647 <== mds.0 v2:172.21.15.161:6836/2559773181 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x1c11260 con 0x1e2a700 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8eb77fe700 10 client.5103 handle_command_reply: tid=0 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 client.5103 shutdown 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 msgr2=0x1e21d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4032000 tx=0x7f8ed401a000 comp rx=0 tx=0).stop 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 shutdown_connections 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x1bdf790 0x1e2a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:6836/2559773181,v1:172.21.15.161:6837/2559773181] conn(0x1e2a700 0x1e6d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x1b94ae0 0x1e29c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 --2- 172.21.15.146:0/3155675647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x1b94710 0x1e21d90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8eb77fe700 0 client.5103 ms_handle_reset on v2:172.21.15.161:6836/2559773181 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 >> 172.21.15.146:0/3155675647 conn(0x1b94eb0 msgr2=0x1dcf890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 shutdown_connections 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 1 -- 172.21.15.146:0/3155675647 wait complete. 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8ee1c2f740 20 client.5103 trim_cache size 0 max 16384 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "cluster_fsid": "10958773-fd73-437e-aa56-f078e9b084bc", 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "whoami": 0, 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 4936, 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "want_state": "up:active", 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "state": "up:active", 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs", 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "rank_uptime": 5.690490466, 2024-08-15T17:00:35.108 INFO:teuthology.orchestra.run.smithi146.stdout: "mdsmap_epoch": 14, 2024-08-15T17:00:35.109 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch": 32, 2024-08-15T17:00:35.109 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_epoch_barrier": 32, 2024-08-15T17:00:35.109 INFO:teuthology.orchestra.run.smithi146.stdout: "uptime": 12.857384822 2024-08-15T17:00:35.109 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:00:35.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 msgr2=0x7f8ebc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 0x7f8ebc062d10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400ba90 tx=0x7f8ed401c000 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 msgr2=0x7f8ed8141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000ea80 tx=0x7f8ed00091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 shutdown_connections 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8ebc060860 0x7f8ebc062d10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ed8155b30 0x7f8ed8141460 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ed8155760 0x7f8ed80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 --2- 172.21.15.146:0/324147305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ed806f250 0x7f8ed8099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 >> 172.21.15.146:0/324147305 conn(0x7f8ed8067810 msgr2=0x7f8ed806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 shutdown_connections 2024-08-15T17:00:35.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.092+0000 7f8edf980700 1 -- 172.21.15.146:0/324147305 wait complete. 2024-08-15T17:00:35.120 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:00:35.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 Processor -- start 2024-08-15T17:00:35.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- start start 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d8159040 0x7f03d80cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 0x7f03d80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80cf190 con 0x7f03d80d0010 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d815d9f0 con 0x7f03d8159040 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d80d09a0 con 0x7f03d815d5b0 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d8159040 0x7f03d80cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f03d8159040 0x7f03d80cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51060/0 (socket says 172.21.15.146:51060) 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 -- 172.21.15.146:0/3956740015 learned_addr learned my addr 172.21.15.146:0/3956740015 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dcd43700 1 --2- 172.21.15.146:0/3956740015 >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 0x7f03d80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 -- 172.21.15.146:0/3956740015 >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 msgr2=0x7f03d80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 --2- 172.21.15.146:0/3956740015 >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 0x7f03d80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 -- 172.21.15.146:0/3956740015 >> v1:172.21.15.161:6789/0 conn(0x7f03d80d0010 legacy=0x7f03d815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 -- 172.21.15.146:0/3956740015 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80ce5e0 con 0x7f03d8159040 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dcd43700 1 --2- 172.21.15.146:0/3956740015 >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 0x7f03d80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d7fff700 1 --2- 172.21.15.146:0/3956740015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d8159040 0x7f03d80cf620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f03d80b6050 tx=0x7f03cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3435c2529a34cf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d6ffd700 1 -- 172.21.15.146:0/3956740015 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00b820 con 0x7f03d8159040 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d6ffd700 1 -- 172.21.15.146:0/3956740015 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00b980 con 0x7f03d8159040 2024-08-15T17:00:35.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03d6ffd700 1 -- 172.21.15.146:0/3956740015 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00bd20 con 0x7f03d8159040 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- 172.21.15.146:0/3956740015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d8159040 msgr2=0x7f03d80cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 --2- 172.21.15.146:0/3956740015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d8159040 0x7f03d80cf620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f03d80b6050 tx=0x7f03cc009250 comp rx=0 tx=0).stop 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- 172.21.15.146:0/3956740015 shutdown_connections 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 --2- 172.21.15.146:0/3956740015 >> v2:172.21.15.161:3300/0 conn(0x7f03d815d5b0 0x7f03d80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 --2- 172.21.15.146:0/3956740015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d8159040 0x7f03d80cf620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- 172.21.15.146:0/3956740015 >> 172.21.15.146:0/3956740015 conn(0x7f03d8068250 msgr2=0x7f03d8068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- 172.21.15.146:0/3956740015 shutdown_connections 2024-08-15T17:00:35.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.220+0000 7f03dd544700 1 -- 172.21.15.146:0/3956740015 wait complete. 2024-08-15T17:00:35.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.224+0000 7f03dd544700 1 Processor -- start 2024-08-15T17:00:35.238 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.224+0000 7f03dd544700 1 -- start start 2024-08-15T17:00:35.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 0x7f03d80afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d815d5b0 0x7f03d8157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d814a180 con 0x7f03d80d0010 2024-08-15T17:00:35.240 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d80b0d30 con 0x7f03d815d5b0 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d814beb0 con 0x7f03d8159040 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51062/0 (socket says 172.21.15.146:51062) 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 -- 172.21.15.146:0/3455815809 learned_addr learned my addr 172.21.15.146:0/3455815809 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dcd43700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d815d5b0 0x7f03d8157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d77fe700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 0x7f03d80afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 -- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 msgr2=0x7f03d80afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 0x7f03d80afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 -- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d815d5b0 msgr2=0x7f03d8157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d815d5b0 0x7f03d8157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 -- 172.21.15.146:0/3455815809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc0094f0 con 0x7f03d80d0010 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d77fe700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 0x7f03d80afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:35.241 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03d7fff700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0030c0 tx=0x7f03cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc011db0 con 0x7f03d80d0010 2024-08-15T17:00:35.242 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc020070 con 0x7f03d80d0010 2024-08-15T17:00:35.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d80b0490 con 0x7f03d80d0010 2024-08-15T17:00:35.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc01b6a0 con 0x7f03d80d0010 2024-08-15T17:00:35.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d80b0a50 con 0x7f03d80d0010 2024-08-15T17:00:35.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.228+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d814bc80 con 0x7f03d80d0010 2024-08-15T17:00:35.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.232+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f03cc022030 con 0x7f03d80d0010 2024-08-15T17:00:35.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.232+0000 7f03bffff700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:35.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.232+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f03cc07e970 con 0x7f03d80d0010 2024-08-15T17:00:35.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.232+0000 7f03d77fe700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:35.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.232+0000 7f03d77fe700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03d80ce280 tx=0x7f03d0009680 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:35.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.240+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f03d814bc80 con 0x7f03d80d0010 2024-08-15T17:00:35.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.440+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f03d814bc80 con 0x7f03d80d0010 2024-08-15T17:00:35.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03bffff700 1 -- 172.21.15.146:0/3455815809 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f03d814bc80 con 0x7f03d80d0010 2024-08-15T17:00:35.455 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 msgr2=0x7f03c4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f03d80ce280 tx=0x7f03d0009680 comp rx=0 tx=0).stop 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 msgr2=0x7f03d8149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0030c0 tx=0x7f03cc016000 comp rx=0 tx=0).stop 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 shutdown_connections 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f03c4060860 0x7f03c4062d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03d815d5b0 0x7f03d8157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03d8159040 0x7f03d80afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 --2- 172.21.15.146:0/3455815809 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80d0010 0x7f03d8149c40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:35.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 >> 172.21.15.146:0/3455815809 conn(0x7f03d8068250 msgr2=0x7f03d80b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:35.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 shutdown_connections 2024-08-15T17:00:35.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:35.444+0000 7f03dd544700 1 -- 172.21.15.146:0/3455815809 wait complete. 2024-08-15T17:00:35.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi146.front.sepia.ceph.com 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:00:35.470 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:00:35.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.471 DEBUG:teuthology.orchestra.run.smithi146:> ip addr 2024-08-15T17:00:35.478 INFO:teuthology.orchestra.run.smithi146.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6e brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:d9:42:6f brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fc brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 0c:c4:7a:88:76:fd brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.479 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.21.15.146/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft 42550sec preferred_lft 42550sec 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::ec4:7aff:fe88:76fd/64 scope link 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether 02:42:1c:9c:24:e8 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether e2:7c:0c:39:5d:35 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::98d9:a9ff:feb5:982a/64 scope link 2024-08-15T17:00:35.480 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether e2:7c:0c:39:5d:35 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::e07c:cff:fe39:5d35/64 scope link 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: link/ether fa:37:f3:27:65:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: inet6 fe80::f837:f3ff:fe27:65f2/64 scope link 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run.smithi146.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:35.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.481 DEBUG:teuthology.orchestra.run.smithi146:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:00:35.493 INFO:teuthology.orchestra.run.smithi146.stdout:1 2024-08-15T17:00:35.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.496 DEBUG:teuthology.orchestra.run.smithi146:> route 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:Kernel IP routing table 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:00:35.547 INFO:teuthology.orchestra.run.smithi146.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:00:35.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> (cd / && exec bash -c ' 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> set -e 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:00:35.548 DEBUG:teuthology.orchestra.run.smithi146:> ') 2024-08-15T17:00:35.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.586 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-08-15T17:00:35.594 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T17:00:35.594 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:00:35.594 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:00:35.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:35.595 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link set brx.0 up 2024-08-15T17:00:35.651 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-08-15T17:00:35.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:35.652 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:35.657 INFO:teuthology.orchestra.run.smithi146.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-15T17:00:35.657 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:35.657 DEBUG:teuthology.orchestra.run.smithi146:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:35.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:35.720 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:35.721 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:35.735 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:35.735 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:35.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:35.736 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:35.743 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:00:35.743 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-08-15T17:00:35.746 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:35.763 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:35.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:35.822 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:35.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:35.824 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:35.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:2024-08-15T17:00:35.816+0000 7f724f327380 -1 init, newargv = 0x55ad60196b10 newargc=15 2024-08-15T17:00:35.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33510]: starting ceph client 2024-08-15T17:00:35.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33510]: starting fuse 2024-08-15T17:00:36.930 DEBUG:teuthology.orchestra.run.smithi146:> sudo modprobe fuse 2024-08-15T17:00:36.946 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:36.946 DEBUG:teuthology.orchestra.run.smithi146:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:36.960 INFO:teuthology.orchestra.run.smithi146.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:36.960 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:36.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:36.961 DEBUG:teuthology.orchestra.run.smithi146:> ls /sys/fs/fuse/connections 2024-08-15T17:00:37.009 INFO:teuthology.orchestra.run.smithi146.stdout:51 2024-08-15T17:00:37.009 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [51] 2024-08-15T17:00:37.009 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:00:37.009 DEBUG:teuthology.orchestra.run.smithi146:> import glob 2024-08-15T17:00:37.009 DEBUG:teuthology.orchestra.run.smithi146:> import re 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> import os 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> import subprocess 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> def _find_admin_socket(client_name): 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> files = glob.glob(asok_path) 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> # Given a non-glob path, it better be there 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> if "*" not in asok_path: 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> assert(len(files) == 1) 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> return files[0] 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:37.010 DEBUG:teuthology.orchestra.run.smithi146:> for f in files: 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> contents = proc_f.read() 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> if mountpoint in contents: 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> return f 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> print(_find_admin_socket("client.0")) 2024-08-15T17:00:37.011 DEBUG:teuthology.orchestra.run.smithi146:> ' 2024-08-15T17:00:37.095 INFO:teuthology.orchestra.run.smithi146.stdout:/var/run/ceph/ceph-client.0.33510.asok 2024-08-15T17:00:37.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33510.asok 2024-08-15T17:00:37.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:37.279 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33510.asok status 2024-08-15T17:00:37.377 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:00:37.377 INFO:teuthology.orchestra.run.smithi146.stdout: "metadata": { 2024-08-15T17:00:37.377 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "entity_id": "0", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "hostname": "smithi146", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "pid": "33510", 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "root": "/" 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_count": 0, 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "id": 5115, 2024-08-15T17:00:37.378 INFO:teuthology.orchestra.run.smithi146.stdout: "inst": { 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "name": { 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "client", 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "num": 5115 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3709524936 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": { 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "type": "v1", 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:37.379 INFO:teuthology.orchestra.run.smithi146.stdout: "nonce": 3709524936 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "inst_str": "client.5115 192.168.0.1:0/3709524936", 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "addr_str": "192.168.0.1:0/3709524936", 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "inode_count": 1, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_epoch": 14, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch": 32, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "blocklisted": false, 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout: "fs_name": "cephfs" 2024-08-15T17:00:37.380 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:00:37.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:37.387 DEBUG:teuthology.orchestra.run.smithi146:> stat --file-system '--printf=%T 2024-08-15T17:00:37.387 DEBUG:teuthology.orchestra.run.smithi146:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:37.395 INFO:teuthology.orchestra.run.smithi146.stdout:fuseblk 2024-08-15T17:00:37.395 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:37.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:37.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-15T17:00:37.408 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-15T17:00:37.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.508+0000 7f880ed73700 1 Processor -- start 2024-08-15T17:00:37.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.508+0000 7f880ed73700 1 -- start start 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880ed73700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808159fa0 0x7f8808155770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880ed73700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88080ca400 con 0x7f8808159bd0 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880ed73700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88080c97f0 con 0x7f8808155cb0 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880ed73700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88080c9ad0 con 0x7f8808159fa0 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880dd71700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f8808159bd0 0x7f88080aee20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45444/0 (socket says 172.21.15.146:45444) 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880dd71700 1 -- 172.21.15.146:0/2197345199 learned_addr learned my addr 172.21.15.146:0/2197345199 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:37.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880d570700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808159fa0 0x7f8808155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724949962 0 0) 0x7f88080ca400 con 0x7f8808159bd0 2024-08-15T17:00:37.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f4003570 con 0x7f8808159bd0 2024-08-15T17:00:37.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610926403 0 0) 0x7f88080c97f0 con 0x7f8808155cb0 2024-08-15T17:00:37.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88080ca400 con 0x7f8808155cb0 2024-08-15T17:00:37.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078415716 0 0) 0x7f87f4003570 con 0x7f8808159bd0 2024-08-15T17:00:37.528 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808159fa0 msgr2=0x7f8808155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808159fa0 0x7f8808155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 >> v1:172.21.15.161:6789/0 conn(0x7f8808155cb0 legacy=0x7f88080ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080c9db0 con 0x7f8808159bd0 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f8804001470 con 0x7f8808159bd0 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 >> v1:172.21.15.146:6789/0 conn(0x7f8808159bd0 legacy=0x7f88080aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87f4003c20 0x7f87f4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4004810 0x7f87f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87f40051c0 0x7f87f400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88080c9ad0 con 0x7f87f4004810 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f400c0b0 con 0x7f87f4003c20 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f400c270 con 0x7f87f40051c0 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880d570700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808159fa0 0x7f8808155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:37.529 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880dd71700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87f4003c20 0x7f87f4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880e572700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87f40051c0 0x7f87f400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.512+0000 7f880d570700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4004810 0x7f87f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 -- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87f40051c0 msgr2=0x7f87f400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87f40051c0 0x7f87f400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 -- 172.21.15.146:0/2197345199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4004810 msgr2=0x7f87f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4004810 0x7f87f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 -- 172.21.15.146:0/2197345199 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4003570 con 0x7f87f4003c20 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880d570700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4004810 0x7f87f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880dd71700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87f4003c20 0x7f87f4004010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f880400c210 tx=0x7f8804001d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd5efad197e7781 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880400cc10 con 0x7f87f4003c20 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880400cd70 con 0x7f87f4003c20 2024-08-15T17:00:37.530 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880cd6f700 1 -- 172.21.15.146:0/2197345199 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804005b70 con 0x7f87f4003c20 2024-08-15T17:00:37.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87f4003c20 msgr2=0x7f87f4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 --2- 172.21.15.146:0/2197345199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87f4003c20 0x7f87f4004010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f880400c210 tx=0x7f8804001d40 comp rx=0 tx=0).stop 2024-08-15T17:00:37.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880e572700 1 -- 172.21.15.146:0/2197345199 reap_dead start 2024-08-15T17:00:37.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- 172.21.15.146:0/2197345199 shutdown_connections 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- 172.21.15.146:0/2197345199 >> 172.21.15.146:0/2197345199 conn(0x7f880806e100 msgr2=0x7f88080b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- 172.21.15.146:0/2197345199 shutdown_connections 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- 172.21.15.146:0/2197345199 wait complete. 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 Processor -- start 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.516+0000 7f880ed73700 1 -- start start 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 0x7f88080ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 0x7f8808067a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8808067fc0 con 0x7f87f4003c20 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8808069570 con 0x7f88080ae2d0 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88080a9e40 con 0x7f8808067610 2024-08-15T17:00:37.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:51084/0 (socket says 172.21.15.146:51084) 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 -- 172.21.15.146:0/1139180962 learned_addr learned my addr 172.21.15.146:0/1139180962 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880e572700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 0x7f8808067a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880d570700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 0x7f88080ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 -- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 msgr2=0x7f8808067a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 0x7f8808067a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 -- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 msgr2=0x7f88080ab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 0x7f88080ab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 -- 172.21.15.146:0/1139180962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080aa120 con 0x7f87f4003c20 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880d570700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 0x7f88080ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880e572700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 0x7f8808067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880dd71700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f880400c800 tx=0x7f8804015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804001d40 con 0x7f87f4003c20 2024-08-15T17:00:37.535 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88040070d0 con 0x7f87f4003c20 2024-08-15T17:00:37.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804007470 con 0x7f87f4003c20 2024-08-15T17:00:37.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88080aa400 con 0x7f87f4003c20 2024-08-15T17:00:37.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88080ae740 con 0x7f87f4003c20 2024-08-15T17:00:37.536 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88080638f0 con 0x7f87f4003c20 2024-08-15T17:00:37.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.520+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8804019030 con 0x7f87f4003c20 2024-08-15T17:00:37.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.524+0000 7f87fdffb700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 0x7f87ec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:37.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.524+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f880407dbf0 con 0x7f87f4003c20 2024-08-15T17:00:37.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.524+0000 7f880d570700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 0x7f87ec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:37.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.524+0000 7f880d570700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 0x7f87ec062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009fd0 tx=0x7f87f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:37.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.528+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f880404e040 con 0x7f87f4003c20 2024-08-15T17:00:37.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8808069340 con 0x7f87f4003c20 2024-08-15T17:00:37.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f87fdffb700 1 -- 172.21.15.146:0/1139180962 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8804051660 con 0x7f87f4003c20 2024-08-15T17:00:37.744 INFO:teuthology.orchestra.run.smithi146.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 msgr2=0x7f87ec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 0x7f87ec062d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009fd0 tx=0x7f87f8008040 comp rx=0 tx=0).stop 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 msgr2=0x7f88080a9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f880400c800 tx=0x7f8804015000 comp rx=0 tx=0).stop 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 shutdown_connections 2024-08-15T17:00:37.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f87ec060890 0x7f87ec062d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8808067610 0x7f8808067a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88080ae2d0 0x7f88080ab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.732+0000 7f880ed73700 1 --2- 172.21.15.146:0/1139180962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87f4003c20 0x7f88080a9860 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.736+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 >> 172.21.15.146:0/1139180962 conn(0x7f880806e100 msgr2=0x7f8808157eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.736+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 shutdown_connections 2024-08-15T17:00:37.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:37.736+0000 7f880ed73700 1 -- 172.21.15.146:0/1139180962 wait complete. 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-15T17:00:37.759 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:00:37.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.760 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:37.768 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42556sec preferred_lft 42556sec 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 02:42:7f:7e:3a:c4 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-08-15T17:00:37.769 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 36:5a:c7:61:1d:25 brd ff:ff:ff:ff:ff:ff 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::4b6:5cff:fe2f:5508/64 scope link 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 36:5a:c7:61:1d:25 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::345a:c7ff:fe61:1d25/64 scope link 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether e6:76:e1:ad:ef:0d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:00:37.770 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::e476:e1ff:fead:ef0d/64 scope link 2024-08-15T17:00:37.771 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-08-15T17:00:37.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.771 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-15T17:00:37.826 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-08-15T17:00:37.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.828 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-15T17:00:37.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-15T17:00:37.835 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-08-15T17:00:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.878 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-08-15T17:00:37.884 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T17:00:37.884 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:00:37.885 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:00:37.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:00:37.885 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-08-15T17:00:37.942 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-08-15T17:00:37.943 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:37.943 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:37.948 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-15T17:00:37.949 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-15T17:00:37.949 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:37.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:38.015 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:38.016 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:38.029 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:38.030 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:38.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:38.030 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:38.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-15T17:00:38.036 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-08-15T17:00:38.080 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:38.098 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:38.098 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:38.109 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:38.110 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:38.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:38.110 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:38.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:2024-08-15T17:00:38.154+0000 7feae1df1380 -1 init, newargv = 0x5639026105f0 newargc=15 2024-08-15T17:00:38.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:ceph-fuse[29427]: starting ceph client 2024-08-15T17:00:38.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:ceph-fuse[29427]: starting fuse 2024-08-15T17:00:39.266 DEBUG:teuthology.orchestra.run.smithi161:> sudo modprobe fuse 2024-08-15T17:00:39.285 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-08-15T17:00:39.285 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-08-15T17:00:39.299 INFO:teuthology.orchestra.run.smithi161.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-08-15T17:00:39.300 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-08-15T17:00:39.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:39.300 DEBUG:teuthology.orchestra.run.smithi161:> ls /sys/fs/fuse/connections 2024-08-15T17:00:39.349 INFO:teuthology.orchestra.run.smithi161.stdout:52 2024-08-15T17:00:39.349 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> import glob 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> import re 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> import os 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> import subprocess 2024-08-15T17:00:39.349 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> def _find_admin_socket(client_name): 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> files = glob.glob(asok_path) 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> # Given a non-glob path, it better be there 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> if "*" not in asok_path: 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> assert(len(files) == 1) 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> return files[0] 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:39.350 DEBUG:teuthology.orchestra.run.smithi161:> for f in files: 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> contents = proc_f.read() 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> if mountpoint in contents: 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> return f 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> print(_find_admin_socket("client.1")) 2024-08-15T17:00:39.351 DEBUG:teuthology.orchestra.run.smithi161:> ' 2024-08-15T17:00:39.437 INFO:teuthology.orchestra.run.smithi161.stdout:/var/run/ceph/ceph-client.1.29427.asok 2024-08-15T17:00:39.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.29427.asok 2024-08-15T17:00:39.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:39.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.29427.asok status 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "metadata": { 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_sha1": "2c991c196a651f8ef0f36c7763ec87a241b25549", 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "ceph_version": "ceph version 17.2.7-1375-g2c991c19 (2c991c196a651f8ef0f36c7763ec87a241b25549) quincy (stable)", 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "entity_id": "1", 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "hostname": "smithi161", 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-08-15T17:00:39.716 INFO:teuthology.orchestra.run.smithi161.stdout: "pid": "29427", 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "root": "/" 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_count": 0, 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "dentry_pinned_count": 0, 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "id": 4895, 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "inst": { 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "name": { 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "client", 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "num": 4895 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:39.717 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4038003622 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: } 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": { 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "type": "v1", 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "addr": "192.168.0.1:0", 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "nonce": 4038003622 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: }, 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inst_str": "client.4895 192.168.0.1:0/4038003622", 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "addr_str": "192.168.0.1:0/4038003622", 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "inode_count": 1, 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "mds_epoch": 14, 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch": 32, 2024-08-15T17:00:39.718 INFO:teuthology.orchestra.run.smithi161.stdout: "osd_epoch_barrier": 0, 2024-08-15T17:00:39.719 INFO:teuthology.orchestra.run.smithi161.stdout: "blocklisted": false, 2024-08-15T17:00:39.719 INFO:teuthology.orchestra.run.smithi161.stdout: "fs_name": "cephfs" 2024-08-15T17:00:39.719 INFO:teuthology.orchestra.run.smithi161.stdout:} 2024-08-15T17:00:39.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:39.725 DEBUG:teuthology.orchestra.run.smithi161:> stat --file-system '--printf=%T 2024-08-15T17:00:39.725 DEBUG:teuthology.orchestra.run.smithi161:> ' -- /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:39.735 INFO:teuthology.orchestra.run.smithi161.stdout:fuseblk 2024-08-15T17:00:39.736 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:39.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-15T17:00:39.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-15T17:00:39.750 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-08-15T17:00:39.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.860+0000 7f74bcd8f700 1 Processor -- start 2024-08-15T17:00:39.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.860+0000 7f74bcd8f700 1 -- start start 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74b80cd2b0 0x7f74b80cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f74b80ceb70 0x7f74b8162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80cc480 con 0x7f74b80caeb0 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b8159720 con 0x7f74b80cd2b0 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80cb870 con 0x7f74b80ceb70 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74b80cd2b0 0x7f74b80cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.876 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f74b80cd2b0 0x7f74b80cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34008/0 (socket says 172.21.15.146:34008) 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 -- 172.21.15.146:0/564030309 learned_addr learned my addr 172.21.15.146:0/564030309 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b6d9d700 1 --2- 172.21.15.146:0/564030309 >> v2:172.21.15.161:3300/0 conn(0x7f74b80ceb70 0x7f74b8162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 -- 172.21.15.146:0/564030309 >> v2:172.21.15.161:3300/0 conn(0x7f74b80ceb70 msgr2=0x7f74b8162ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 --2- 172.21.15.146:0/564030309 >> v2:172.21.15.161:3300/0 conn(0x7f74b80ceb70 0x7f74b8162ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 -- 172.21.15.146:0/564030309 >> v1:172.21.15.161:6789/0 conn(0x7f74b80caeb0 legacy=0x7f74b80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:39.877 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 -- 172.21.15.146:0/564030309 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b80cbb50 con 0x7f74b80cd2b0 2024-08-15T17:00:39.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b659c700 1 --2- 172.21.15.146:0/564030309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80cd2b0 0x7f74b80cd680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74b8099c60 tx=0x7f74a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8acb30501a568a server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:39.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b559a700 1 -- 172.21.15.146:0/564030309 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a000b860 con 0x7f74b80cd2b0 2024-08-15T17:00:39.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b559a700 1 -- 172.21.15.146:0/564030309 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a000b9c0 con 0x7f74b80cd2b0 2024-08-15T17:00:39.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74b559a700 1 -- 172.21.15.146:0/564030309 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a000bd60 con 0x7f74b80cd2b0 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- 172.21.15.146:0/564030309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80cd2b0 msgr2=0x7f74b80cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/564030309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80cd2b0 0x7f74b80cd680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f74b8099c60 tx=0x7f74a0009250 comp rx=0 tx=0).stop 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- 172.21.15.146:0/564030309 shutdown_connections 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/564030309 >> v2:172.21.15.161:3300/0 conn(0x7f74b80ceb70 0x7f74b8162ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/564030309 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80cd2b0 0x7f74b80cd680 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- 172.21.15.146:0/564030309 >> 172.21.15.146:0/564030309 conn(0x7f74b8099550 msgr2=0x7f74b8067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- 172.21.15.146:0/564030309 shutdown_connections 2024-08-15T17:00:39.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.864+0000 7f74bcd8f700 1 -- 172.21.15.146:0/564030309 wait complete. 2024-08-15T17:00:39.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 Processor -- start 2024-08-15T17:00:39.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 -- start start 2024-08-15T17:00:39.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74b80caeb0 0x7f74b80c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74b80cd2b0 0x7f74b8158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80c9050 con 0x7f74b80ceb70 2024-08-15T17:00:39.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80c9860 con 0x7f74b80caeb0 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74bcd8f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80c7860 con 0x7f74b80cd2b0 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34010/0 (socket says 172.21.15.146:34010) 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 -- 172.21.15.146:0/3969620326 learned_addr learned my addr 172.21.15.146:0/3969620326 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b659c700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74b80caeb0 0x7f74b80c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b5d9b700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74b80cd2b0 0x7f74b8158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 -- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74b80cd2b0 msgr2=0x7f74b8158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74b80cd2b0 0x7f74b8158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 -- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74b80caeb0 msgr2=0x7f74b80c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74b80caeb0 0x7f74b80c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.868+0000 7f74b6d9d700 1 -- 172.21.15.146:0/3969620326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a00094f0 con 0x7f74b80ceb70 2024-08-15T17:00:39.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74b6d9d700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74b0007f60 tx=0x7f74b00098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000eb90 con 0x7f74b80ceb70 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b000ecf0 con 0x7f74b80ceb70 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b80c7b40 con 0x7f74b80ceb70 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0014670 con 0x7f74b80ceb70 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b80c8100 con 0x7f74b80ceb70 2024-08-15T17:00:39.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b80638f0 con 0x7f74b80ceb70 2024-08-15T17:00:39.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f74b0014a60 con 0x7f74b80ceb70 2024-08-15T17:00:39.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 0x7f7498062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:39.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74b007d1a0 con 0x7f74b80ceb70 2024-08-15T17:00:39.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.872+0000 7f74b659c700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 0x7f7498062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:39.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.876+0000 7f74b659c700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 0x7f7498062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f74b8099c60 tx=0x7f74a0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:39.893 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:39.880+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f74b004d5f0 con 0x7f74b80ceb70 2024-08-15T17:00:40.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.084+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f74b8068020 con 0x7f74b80ceb70 2024-08-15T17:00:40.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.736+0000 7f74ae7fc700 1 -- 172.21.15.146:0/3969620326 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 136+0+0 (secure 0 0 0) 0x7f74b0050c10 con 0x7f74b80ceb70 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 msgr2=0x7f7498062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 0x7f7498062d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f74b8099c60 tx=0x7f74a0017000 comp rx=0 tx=0).stop 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 msgr2=0x7f74b8157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74b0007f60 tx=0x7f74b00098e0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 shutdown_connections 2024-08-15T17:00:40.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7498060860 0x7f7498062d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f74b80ceb70 0x7f74b8157b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f74b80cd2b0 0x7f74b8158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 --2- 172.21.15.146:0/3969620326 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f74b80caeb0 0x7f74b80c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 >> 172.21.15.146:0/3969620326 conn(0x7f74b8099550 msgr2=0x7f74b815bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 shutdown_connections 2024-08-15T17:00:40.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.740+0000 7f74bcd8f700 1 -- 172.21.15.146:0/3969620326 wait complete. 2024-08-15T17:00:40.769 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:00:40.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 Processor -- start 2024-08-15T17:00:40.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 -- start start 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9204155d20 0x7f92040b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f92040aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040af2b0 con 0x7f920406df10 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92040adc10 con 0x7f9204155d20 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920adfb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040af590 con 0x7f92040ad7c0 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920a5fa700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f920406df10 0x7f92041599c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46700/0 (socket says 172.21.15.146:46700) 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920a5fa700 1 -- 172.21.15.146:0/2679564159 learned_addr learned my addr 172.21.15.146:0/2679564159 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9209df9700 1 --2- 172.21.15.146:0/2679564159 >> v2:172.21.15.161:3300/0 conn(0x7f9204155d20 0x7f92040b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f92095f8700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f92040aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.886 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963575125 0 0) 0x7f92040af2b0 con 0x7f920406df10 2024-08-15T17:00:40.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ec003570 con 0x7f920406df10 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 150197958 0 0) 0x7f91ec003570 con 0x7f920406df10 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 msgr2=0x7f92040aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f92040aea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 >> v2:172.21.15.161:3300/0 conn(0x7f9204155d20 msgr2=0x7f92040b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 --2- 172.21.15.146:0/2679564159 >> v2:172.21.15.161:3300/0 conn(0x7f9204155d20 0x7f92040b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040af870 con 0x7f920406df10 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f91fc001470 con 0x7f920406df10 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 >> v1:172.21.15.146:6789/0 conn(0x7f920406df10 legacy=0x7f92041599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f91ec003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91ec003fe0 0x7f91ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91ec004d50 0x7f91ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040adc10 con 0x7f91ec004d50 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040af590 con 0x7f91ec003fe0 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ec00dca0 con 0x7f92040ad7c0 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.872+0000 7f920a5fa700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91ec004d50 0x7f91ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f92095f8700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91ec003fe0 0x7f91ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f9209df9700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f91ec003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 -- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 msgr2=0x7f91ec003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:40.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f91ec003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 -- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91ec003fe0 msgr2=0x7f91ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91ec003fe0 0x7f91ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 -- 172.21.15.146:0/2679564159 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ec003570 con 0x7f91ec004d50 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f9209df9700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92040ad7c0 0x7f91ec003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91ec004d50 0x7f91ec0051c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c3c0 tx=0x7f91fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab685a67f292f75 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00cc30 con 0x7f91ec004d50 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc00cd90 con 0x7f91ec004d50 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f9208df7700 1 -- 172.21.15.146:0/2679564159 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc005b70 con 0x7f91ec004d50 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 -- 172.21.15.146:0/2679564159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91ec004d50 msgr2=0x7f91ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 --2- 172.21.15.146:0/2679564159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f91ec004d50 0x7f91ec0051c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c3c0 tx=0x7f91fc001e90 comp rx=0 tx=0).stop 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920a5fa700 1 -- 172.21.15.146:0/2679564159 reap_dead start 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 -- 172.21.15.146:0/2679564159 shutdown_connections 2024-08-15T17:00:40.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 -- 172.21.15.146:0/2679564159 >> 172.21.15.146:0/2679564159 conn(0x7f92040678d0 msgr2=0x7f9204067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:40.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 -- 172.21.15.146:0/2679564159 shutdown_connections 2024-08-15T17:00:40.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.876+0000 7f920adfb700 1 -- 172.21.15.146:0/2679564159 wait complete. 2024-08-15T17:00:40.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 Processor -- start 2024-08-15T17:00:40.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 -- start start 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91ec004d50 0x7f9204146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92041588c0 0x7f9204151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041481f0 con 0x7f92041515b0 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041475e0 con 0x7f92041588c0 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920adfb700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92041478c0 con 0x7f91ec004d50 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f9209df9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91ec004d50 0x7f9204146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34018/0 (socket says 172.21.15.146:34018) 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 -- 172.21.15.146:0/3513700683 learned_addr learned my addr 172.21.15.146:0/3513700683 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f92095f8700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92041588c0 0x7f9204151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 -- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91ec004d50 msgr2=0x7f9204146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91ec004d50 0x7f9204146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 -- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92041588c0 msgr2=0x7f9204151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92041588c0 0x7f9204151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 -- 172.21.15.146:0/3513700683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204147ba0 con 0x7f92041515b0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.880+0000 7f920a5fa700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009020 tx=0x7f91fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc00cc30 con 0x7f92041515b0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91fc001430 con 0x7f92041515b0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9204147e80 con 0x7f92041515b0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91fc012400 con 0x7f92041515b0 2024-08-15T17:00:40.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9204148bc0 con 0x7f92041515b0 2024-08-15T17:00:40.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9204148360 con 0x7f92041515b0 2024-08-15T17:00:40.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f91fc019030 con 0x7f92041515b0 2024-08-15T17:00:40.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 0x7f91f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:40.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f91fc07d840 con 0x7f92041515b0 2024-08-15T17:00:40.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.884+0000 7f9209df9700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 0x7f91f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:40.899 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.888+0000 7f9209df9700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 0x7f91f0062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009270 tx=0x7f91f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:40.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:40.892+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f91fc04dc70 con 0x7f92041515b0 2024-08-15T17:00:41.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.080+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f92040643a0 con 0x7f91f0060860 2024-08-15T17:00:41.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f91f9ffb700 1 -- 172.21.15.146:0/3513700683 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f92040643a0 con 0x7f91f0060860 2024-08-15T17:00:41.096 INFO:teuthology.orchestra.run.smithi146.stdout:[ 2024-08-15T17:00:41.096 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:41.096 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "cephfs" 2024-08-15T17:00:41.096 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:41.096 INFO:teuthology.orchestra.run.smithi146.stdout:] 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 msgr2=0x7f91f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 0x7f91f0062d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009270 tx=0x7f91f4008040 comp rx=0 tx=0).stop 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 msgr2=0x7f9204148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009020 tx=0x7f91fc015000 comp rx=0 tx=0).stop 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 shutdown_connections 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f91f0060860 0x7f91f0062d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92041515b0 0x7f9204148680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92041588c0 0x7f9204151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 --2- 172.21.15.146:0/3513700683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91ec004d50 0x7f9204146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 >> 172.21.15.146:0/3513700683 conn(0x7f92040678d0 msgr2=0x7f92041534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 shutdown_connections 2024-08-15T17:00:41.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.084+0000 7f920adfb700 1 -- 172.21.15.146:0/3513700683 wait complete. 2024-08-15T17:00:41.109 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-15T17:00:41.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.204+0000 7f264b5f2700 1 Processor -- start 2024-08-15T17:00:41.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.204+0000 7f264b5f2700 1 -- start start 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2644159300 0x7f26440b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f26441596d0 0x7f2644066e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26440cba90 con 0x7f264415dbc0 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440b2b60 con 0x7f2644159300 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440cb160 con 0x7f26441596d0 2024-08-15T17:00:41.220 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2649b8f700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f264415dbc0 0x7f26440afe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46708/0 (socket says 172.21.15.146:46708) 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2649b8f700 1 -- 172.21.15.146:0/920667671 learned_addr learned my addr 172.21.15.146:0/920667671 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 --2- 172.21.15.146:0/920667671 >> v2:172.21.15.146:3300/0 conn(0x7f2644159300 0x7f26440b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2648b8d700 1 --2- 172.21.15.146:0/920667671 >> v2:172.21.15.161:3300/0 conn(0x7f26441596d0 0x7f2644066e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2637fff700 1 -- 172.21.15.146:0/920667671 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197352023 0 0) 0x7f26440cba90 con 0x7f264415dbc0 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2637fff700 1 -- 172.21.15.146:0/920667671 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2630003570 con 0x7f264415dbc0 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 -- 172.21.15.146:0/920667671 >> v2:172.21.15.161:3300/0 conn(0x7f26441596d0 msgr2=0x7f2644066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 --2- 172.21.15.146:0/920667671 >> v2:172.21.15.161:3300/0 conn(0x7f26441596d0 0x7f2644066e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 -- 172.21.15.146:0/920667671 >> v1:172.21.15.146:6789/0 conn(0x7f264415dbc0 legacy=0x7f26440afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 -- 172.21.15.146:0/920667671 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440cb440 con 0x7f2644159300 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264938e700 1 --2- 172.21.15.146:0/920667671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2644159300 0x7f26440b2230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26440b2d20 tx=0x7f26380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34b791012c9116 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2637fff700 1 -- 172.21.15.146:0/920667671 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638011630 con 0x7f2644159300 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2637fff700 1 -- 172.21.15.146:0/920667671 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638011790 con 0x7f2644159300 2024-08-15T17:00:41.221 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f2637fff700 1 -- 172.21.15.146:0/920667671 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638011b30 con 0x7f2644159300 2024-08-15T17:00:41.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- 172.21.15.146:0/920667671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2644159300 msgr2=0x7f26440b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 --2- 172.21.15.146:0/920667671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2644159300 0x7f26440b2230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f26440b2d20 tx=0x7f26380091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- 172.21.15.146:0/920667671 shutdown_connections 2024-08-15T17:00:41.222 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 --2- 172.21.15.146:0/920667671 >> v2:172.21.15.161:3300/0 conn(0x7f26441596d0 0x7f2644066e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 --2- 172.21.15.146:0/920667671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2644159300 0x7f26440b2230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- 172.21.15.146:0/920667671 >> 172.21.15.146:0/920667671 conn(0x7f26440682c0 msgr2=0x7f26440686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- 172.21.15.146:0/920667671 shutdown_connections 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.208+0000 7f264b5f2700 1 -- 172.21.15.146:0/920667671 wait complete. 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 Processor -- start 2024-08-15T17:00:41.223 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- start start 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2644159300 0x7f26441500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 0x7f264414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440c4a20 con 0x7f26441596d0 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26440c3e50 con 0x7f264415dbc0 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26440c40f0 con 0x7f2644159300 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34038/0 (socket says 172.21.15.146:34038) 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 -- 172.21.15.146:0/3568811478 learned_addr learned my addr 172.21.15.146:0/3568811478 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264938e700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2644159300 0x7f26441500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2649b8f700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 0x7f264414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.226 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 -- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2644159300 msgr2=0x7f26441500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2644159300 0x7f26441500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 -- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 msgr2=0x7f264414dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 0x7f264414dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 -- 172.21.15.146:0/3568811478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638009420 con 0x7f26441596d0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2649b8f700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 0x7f264414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f2648b8d700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f264000d890 tx=0x7f2640008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400125f0 con 0x7f26441596d0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2640012750 con 0x7f26441596d0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640012ad0 con 0x7f26441596d0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440c43d0 con 0x7f26441596d0 2024-08-15T17:00:41.227 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.212+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264414cc60 con 0x7f26441596d0 2024-08-15T17:00:41.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26440c4bd0 con 0x7f26441596d0 2024-08-15T17:00:41.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f264001e460 con 0x7f26441596d0 2024-08-15T17:00:41.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f26357fa700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 0x7f2628062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f264007d1d0 con 0x7f26441596d0 2024-08-15T17:00:41.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f264938e700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 0x7f2628062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.216+0000 7f264938e700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 0x7f2628062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f26380093f0 tx=0x7f2638017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.224+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f26440c4bd0 con 0x7f26441596d0 2024-08-15T17:00:41.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.392+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f26440645c0 con 0x7f26441596d0 2024-08-15T17:00:41.406 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.392+0000 7f26357fa700 1 -- 172.21.15.146:0/3568811478 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f26400510b0 con 0x7f26441596d0 2024-08-15T17:00:41.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 msgr2=0x7f2628062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 0x7f2628062d10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f26380093f0 tx=0x7f2638017000 comp rx=0 tx=0).stop 2024-08-15T17:00:41.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 msgr2=0x7f2644151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f264000d890 tx=0x7f2640008040 comp rx=0 tx=0).stop 2024-08-15T17:00:41.409 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 shutdown_connections 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2628060860 0x7f2628062d10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f264415dbc0 0x7f264414dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f26441596d0 0x7f2644151d10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 --2- 172.21.15.146:0/3568811478 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2644159300 0x7f26441500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 >> 172.21.15.146:0/3568811478 conn(0x7f26440682c0 msgr2=0x7f26440b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 shutdown_connections 2024-08-15T17:00:41.410 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.396+0000 7f264b5f2700 1 -- 172.21.15.146:0/3568811478 wait complete. 2024-08-15T17:00:41.421 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000940580 2024-08-15T17:00:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 Processor -- start 2024-08-15T17:00:41.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 -- start start 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54a80b2210 0x7f54a80b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a80b5a00 con 0x7f54a80cc840 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a80af570 con 0x7f54a80b2210 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54ad0b6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8159ce0 con 0x7f54a80b5040 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a659c700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f54a80b2210 0x7f54a80b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a759e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:55146/0 (socket says 172.21.15.146:55146) 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a759e700 1 -- 172.21.15.146:0/3432046263 learned_addr learned my addr 172.21.15.146:0/3432046263 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:41.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a6d9d700 1 --1- 172.21.15.146:0/3432046263 >> v1:172.21.15.161:6789/0 conn(0x7f54a80cc840 0x7f54a80ccc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:45908/0 (socket says 172.21.15.146:45908) 2024-08-15T17:00:41.541 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a659c700 1 -- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 msgr2=0x7f54a80b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a659c700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a659c700 1 -- 172.21.15.146:0/3432046263 >> v1:172.21.15.161:6789/0 conn(0x7f54a80cc840 legacy=0x7f54a80ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a5d9b700 1 -- 172.21.15.146:0/3432046263 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204088863 0 0) 0x7f54a80b5a00 con 0x7f54a80cc840 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a659c700 1 -- 172.21.15.146:0/3432046263 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8159fc0 con 0x7f54a80b2210 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.520+0000 7f54a759e700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54a659c700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a80b2670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f549c0098e0 tx=0x7f549c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0a23f95b57337cc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54a5d9b700 1 -- 172.21.15.146:0/3432046263 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00b640 con 0x7f54a80b2210 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54a5d9b700 1 -- 172.21.15.146:0/3432046263 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00b7a0 con 0x7f54a80b2210 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54a5d9b700 1 -- 172.21.15.146:0/3432046263 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c00bb40 con 0x7f54a80b2210 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 msgr2=0x7f54a80b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a80b2670 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f549c0098e0 tx=0x7f549c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- 172.21.15.146:0/3432046263 shutdown_connections 2024-08-15T17:00:41.542 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80b5040 0x7f54a80b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/3432046263 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a80b2670 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- 172.21.15.146:0/3432046263 >> 172.21.15.146:0/3432046263 conn(0x7f54a80682c0 msgr2=0x7f54a80686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- 172.21.15.146:0/3432046263 shutdown_connections 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- 172.21.15.146:0/3432046263 wait complete. 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 Processor -- start 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.524+0000 7f54ad0b6700 1 -- start start 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a8067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80cc840 0x7f54a816c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8067ab0 con 0x7f54a80b5040 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80c9820 con 0x7f54a80b2210 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80c9b00 con 0x7f54a80cc840 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.543 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34044/0 (socket says 172.21.15.146:34044) 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 -- 172.21.15.146:0/65962539 learned_addr learned my addr 172.21.15.146:0/65962539 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a6d9d700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a8067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a759e700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80cc840 0x7f54a816c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 -- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80cc840 msgr2=0x7f54a816c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80cc840 0x7f54a816c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 -- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 msgr2=0x7f54a8067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a8067570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 -- 172.21.15.146:0/65962539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a0009ae0 con 0x7f54a80b5040 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a6d9d700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a8067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54a659c700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f80 tx=0x7f549c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c000d10 con 0x7f54a80b5040 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c00bb20 con 0x7f54a80b5040 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c009420 con 0x7f54a80b5040 2024-08-15T17:00:41.544 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c011670 con 0x7f54a80b5040 2024-08-15T17:00:41.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a80ca0c0 con 0x7f54a80b5040 2024-08-15T17:00:41.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.528+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a80b5840 con 0x7f54a80b5040 2024-08-15T17:00:41.545 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.532+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f549c011aa0 con 0x7f54a80b5040 2024-08-15T17:00:41.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.532+0000 7f548effd700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 0x7f5494062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:41.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.532+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f549c07e670 con 0x7f54a80b5040 2024-08-15T17:00:41.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.532+0000 7f54a6d9d700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 0x7f5494062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:41.546 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.532+0000 7f54a6d9d700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 0x7f5494062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5498007fd0 tx=0x7f5498007e50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:41.552 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.540+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f54a80b5840 con 0x7f54a80b5040 2024-08-15T17:00:41.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:41.728+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000940580", "target": ["mon-mgr", ""]}) v1 -- 0x7f54a80b5840 con 0x7f5494060860 2024-08-15T17:00:43.860 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.848+0000 7f548effd700 1 -- 172.21.15.146:0/65962539 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f54a80b5840 con 0x7f5494060860 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 msgr2=0x7f5494062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 0x7f5494062d10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5498007fd0 tx=0x7f5498007e50 comp rx=0 tx=0).stop 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 msgr2=0x7f54a80cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f549c009f80 tx=0x7f549c017000 comp rx=0 tx=0).stop 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 shutdown_connections 2024-08-15T17:00:43.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5494060860 0x7f5494062d10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f54a80cc840 0x7f54a816c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f54a80b5040 0x7f54a80cb250 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 --2- 172.21.15.146:0/65962539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f54a80b2210 0x7f54a8067570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 >> 172.21.15.146:0/65962539 conn(0x7f54a80682c0 msgr2=0x7f54a80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 shutdown_connections 2024-08-15T17:00:43.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.856+0000 7f54ad0b6700 1 -- 172.21.15.146:0/65962539 wait complete. 2024-08-15T17:00:43.873 INFO:teuthology.orchestra.run.smithi146.stderr:Volume created successfully (no MDS daemons created) 2024-08-15T17:00:43.887 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-15T17:00:43.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.976+0000 7f10fbed4700 1 Processor -- start 2024-08-15T17:00:43.990 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.976+0000 7f10fbed4700 1 -- start start 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10fbed4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f10f4068db0 0x7f10f40aca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10fbed4700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f40c9520 con 0x7f10f4155ef0 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10fbed4700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f40ad6e0 con 0x7f10f40acfb0 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10fbed4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f406e840 con 0x7f10f4068db0 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f9c70700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f10f4155ef0 0x7f10f40686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46720/0 (socket says 172.21.15.146:46720) 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f946f700 1 --2- 172.21.15.146:0/1372279357 >> v2:172.21.15.146:3300/0 conn(0x7f10f4068db0 0x7f10f40aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f9c70700 1 -- 172.21.15.146:0/1372279357 learned_addr learned my addr 172.21.15.146:0/1372279357 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:43.997 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523610378 0 0) 0x7f10f40c9520 con 0x7f10f4155ef0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e0003570 con 0x7f10f4155ef0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f946f700 1 -- 172.21.15.146:0/1372279357 >> v1:172.21.15.161:6789/0 conn(0x7f10f40acfb0 legacy=0x7f10f40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f946f700 1 -- 172.21.15.146:0/1372279357 >> v1:172.21.15.146:6789/0 conn(0x7f10f4155ef0 legacy=0x7f10f40686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1772555218 0 0) 0x7f10f40ad6e0 con 0x7f10f40acfb0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f946f700 1 -- 172.21.15.146:0/1372279357 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f40ad9c0 con 0x7f10f4068db0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f946f700 1 --2- 172.21.15.146:0/1372279357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4068db0 0x7f10f40aca70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f10e40098e0 tx=0x7f10e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53eb2afd9933283 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e400b640 con 0x7f10f4068db0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e400b7a0 con 0x7f10f4068db0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.980+0000 7f10f8c6e700 1 -- 172.21.15.146:0/1372279357 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e400bb20 con 0x7f10f4068db0 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- 172.21.15.146:0/1372279357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4068db0 msgr2=0x7f10f40aca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 --2- 172.21.15.146:0/1372279357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4068db0 0x7f10f40aca70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f10e40098e0 tx=0x7f10e40091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- 172.21.15.146:0/1372279357 shutdown_connections 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 --2- 172.21.15.146:0/1372279357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4068db0 0x7f10f40aca70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- 172.21.15.146:0/1372279357 >> 172.21.15.146:0/1372279357 conn(0x7f10f40cd160 msgr2=0x7f10f406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- 172.21.15.146:0/1372279357 shutdown_connections 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- 172.21.15.146:0/1372279357 wait complete. 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 Processor -- start 2024-08-15T17:00:43.999 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.984+0000 7f10fbed4700 1 -- start start 2024-08-15T17:00:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10f4068db0 0x7f10f40674a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 0x7f10f40679e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4166650 con 0x7f10f4155ef0 2024-08-15T17:00:44.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4166930 con 0x7f10f40acfb0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4166c10 con 0x7f10f4068db0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34062/0 (socket says 172.21.15.146:34062) 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 -- 172.21.15.146:0/2981429321 learned_addr learned my addr 172.21.15.146:0/2981429321 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10f9c70700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10f4068db0 0x7f10f40674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10f946f700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 0x7f10f40679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 -- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10f4068db0 msgr2=0x7f10f40674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10f4068db0 0x7f10f40674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 -- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 msgr2=0x7f10f40679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 0x7f10f40679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 -- 172.21.15.146:0/2981429321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009420 con 0x7f10f4155ef0 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10f946f700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 0x7f10f40679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:44.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fa471700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10ec001560 tx=0x7f10ec001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003030 con 0x7f10f4155ef0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00a7e0 con 0x7f10f4155ef0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f4166ef0 con 0x7f10f4155ef0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00aba0 con 0x7f10f4155ef0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4166190 con 0x7f10f4155ef0 2024-08-15T17:00:44.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.988+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f41663e0 con 0x7f10f4155ef0 2024-08-15T17:00:44.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.992+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f10ec012420 con 0x7f10f4155ef0 2024-08-15T17:00:44.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.992+0000 7f10e9ffb700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 0x7f10d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.992+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f10ec004070 con 0x7f10f4155ef0 2024-08-15T17:00:44.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.992+0000 7f10f9c70700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 0x7f10d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:43.992+0000 7f10f9c70700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 0x7f10d8062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10f0007fd0 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.011 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.000+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f10ec083080 con 0x7f10f4155ef0 2024-08-15T17:00:44.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.184+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f10f40645c0 con 0x7f10d8060860 2024-08-15T17:00:44.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10e9ffb700 1 -- 172.21.15.146:0/2981429321 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f10f40645c0 con 0x7f10d8060860 2024-08-15T17:00:44.200 INFO:teuthology.orchestra.run.smithi146.stdout:[ 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "cephfs" 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: { 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: "name": "volume_0000000000940580" 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-08-15T17:00:44.201 INFO:teuthology.orchestra.run.smithi146.stdout:] 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 msgr2=0x7f10d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 0x7f10d8062d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f10f0007fd0 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 msgr2=0x7f10f415b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10ec001560 tx=0x7f10ec001770 comp rx=0 tx=0).stop 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 shutdown_connections 2024-08-15T17:00:44.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f10d8060860 0x7f10d8062d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10f4155ef0 0x7f10f415b430 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f10f40acfb0 0x7f10f40679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 --2- 172.21.15.146:0/2981429321 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f10f4068db0 0x7f10f40674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.188+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 >> 172.21.15.146:0/2981429321 conn(0x7f10f40cd160 msgr2=0x7f10f40ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.192+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 shutdown_connections 2024-08-15T17:00:44.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.192+0000 7f10fbed4700 1 -- 172.21.15.146:0/2981429321 wait complete. 2024-08-15T17:00:44.215 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-08-15T17:00:44.326 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.312+0000 7fd8ffaea700 1 Processor -- start 2024-08-15T17:00:44.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.312+0000 7fd8ffaea700 1 -- start start 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8f80cd3d0 0x7fd8f80ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f80cd840 con 0x7fd8f80cf020 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80b2e80 con 0x7fd8f815cb60 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f80cca00 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd886700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd8f80cd3d0 0x7fd8f80ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd886700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34078/0 (socket says 172.21.15.146:34078) 2024-08-15T17:00:44.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd886700 1 -- 172.21.15.146:0/2747351939 learned_addr learned my addr 172.21.15.146:0/2747351939 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fc884700 1 -- 172.21.15.146:0/2747351939 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585297366 0 0) 0x7fd8f80cd840 con 0x7fd8f80cf020 2024-08-15T17:00:44.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fc884700 1 -- 172.21.15.146:0/2747351939 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003570 con 0x7fd8f80cf020 2024-08-15T17:00:44.331 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 -- 172.21.15.146:0/2747351939 >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 msgr2=0x7fd8f80cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 --2- 172.21.15.146:0/2747351939 >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 -- 172.21.15.146:0/2747351939 >> v1:172.21.15.161:6789/0 conn(0x7fd8f80cf020 legacy=0x7fd8f80c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 -- 172.21.15.146:0/2747351939 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f815cf30 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd886700 1 --2- 172.21.15.146:0/2747351939 >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fd085700 1 --2- 172.21.15.146:0/2747351939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f80ceae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8f809e390 tx=0x7fd8f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d04d0f0da6560f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fc884700 1 -- 172.21.15.146:0/2747351939 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4011630 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fc884700 1 -- 172.21.15.146:0/2747351939 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f40117b0 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8fc884700 1 -- 172.21.15.146:0/2747351939 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4011b70 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- 172.21.15.146:0/2747351939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 msgr2=0x7fd8f80ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/2747351939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f80ceae0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd8f809e390 tx=0x7fd8f40091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- 172.21.15.146:0/2747351939 shutdown_connections 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/2747351939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f80ceae0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.333 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/2747351939 >> v2:172.21.15.146:3300/0 conn(0x7fd8f815cb60 0x7fd8f80cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- 172.21.15.146:0/2747351939 >> 172.21.15.146:0/2747351939 conn(0x7fd8f80996d0 msgr2=0x7fd8f80675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- 172.21.15.146:0/2747351939 shutdown_connections 2024-08-15T17:00:44.334 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.316+0000 7fd8ffaea700 1 -- 172.21.15.146:0/2747351939 wait complete. 2024-08-15T17:00:44.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 Processor -- start 2024-08-15T17:00:44.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 -- start start 2024-08-15T17:00:44.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f81480a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8f80cf020 0x7fd8f815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f80c8430 con 0x7fd8f815cb60 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f815bd00 con 0x7fd8f80cd3d0 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8ffaea700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f816a000 con 0x7fd8f80cf020 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34080/0 (socket says 172.21.15.146:34080) 2024-08-15T17:00:44.338 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 -- 172.21.15.146:0/463256581 learned_addr learned my addr 172.21.15.146:0/463256581 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fd886700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f81480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fd085700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8f80cf020 0x7fd8f815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 -- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8f80cf020 msgr2=0x7fd8f815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8f80cf020 0x7fd8f815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 -- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 msgr2=0x7fd8f81480a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f81480a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.320+0000 7fd8fe087700 1 -- 172.21.15.146:0/463256581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80094f0 con 0x7fd8f815cb60 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8fe087700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c940 tx=0x7fd8f00027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.339 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000a740 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f000a8c0 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f4009420 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000ac80 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f81699b0 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f816a190 con 0x7fd8f815cb60 2024-08-15T17:00:44.340 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd8f0012630 con 0x7fd8f815cb60 2024-08-15T17:00:44.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 0x7fd8dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8fd886700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 0x7fd8dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.341 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.324+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fd8f007d330 con 0x7fd8f815cb60 2024-08-15T17:00:44.342 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.328+0000 7fd8fd886700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 0x7fd8dc062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.332+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd8f004d400 con 0x7fd8f815cb60 2024-08-15T17:00:44.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.492+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7fd8f80645c0 con 0x7fd8f815cb60 2024-08-15T17:00:44.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.492+0000 7fd8edffb700 1 -- 172.21.15.146:0/463256581 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v35) v1 ==== 97+0+8050 (secure 0 0 0) 0x7fd8f0050a20 con 0x7fd8f815cb60 2024-08-15T17:00:44.507 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:44.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 msgr2=0x7fd8dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 0x7fd8dc062d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8006fd0 tx=0x7fd8e8008040 comp rx=0 tx=0).stop 2024-08-15T17:00:44.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 msgr2=0x7fd8f815f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c940 tx=0x7fd8f00027a0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 shutdown_connections 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd8dc060860 0x7fd8dc062d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd8f815cb60 0x7fd8f815f790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd8f80cf020 0x7fd8f815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 --2- 172.21.15.146:0/463256581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd8f80cd3d0 0x7fd8f81480a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 >> 172.21.15.146:0/463256581 conn(0x7fd8f80996d0 msgr2=0x7fd8f80b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 shutdown_connections 2024-08-15T17:00:44.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.496+0000 7fd8ffaea700 1 -- 172.21.15.146:0/463256581 wait complete. 2024-08-15T17:00:44.520 INFO:teuthology.orchestra.run.smithi146.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}] 2024-08-15T17:00:44.520 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000940580 --format=json 2024-08-15T17:00:44.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 Processor -- start 2024-08-15T17:00:44.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 -- start start 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f31ec0ad090 0x7f31ec157a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec0ad8a0 con 0x7f31ec155d80 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ec0adb80 con 0x7f31ec0ad090 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f1cd5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0ade60 con 0x7f31ec0af5f0 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f0cd3700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f31ec155d80 0x7f31ec0aeee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:46732/0 (socket says 172.21.15.146:46732) 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f0cd3700 1 -- 172.21.15.146:0/3400488740 learned_addr learned my addr 172.21.15.146:0/3400488740 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 --2- 172.21.15.146:0/3400488740 >> v2:172.21.15.146:3300/0 conn(0x7f31ec0ad090 0x7f31ec157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31ebfff700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112453539 0 0) 0x7f31ec0ad8a0 con 0x7f31ec155d80 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d8003570 con 0x7f31ec155d80 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 -- 172.21.15.146:0/3400488740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 msgr2=0x7f31ec0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 -- 172.21.15.146:0/3400488740 >> v1:172.21.15.146:6789/0 conn(0x7f31ec155d80 legacy=0x7f31ec0aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2763279642 0 0) 0x7f31d8003570 con 0x7f31ec155d80 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 -- 172.21.15.146:0/3400488740 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0ae140 con 0x7f31ec0ad090 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.612+0000 7f31f14d4700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec0ad090 0x7f31ec157a90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009910 tx=0x7f31e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d29b8f027c6d6d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40017f0 con 0x7f31ec0ad090 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4011420 con 0x7f31ec0ad090 2024-08-15T17:00:44.628 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31eb7fe700 1 -- 172.21.15.146:0/3400488740 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40117c0 con 0x7f31ec0ad090 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- 172.21.15.146:0/3400488740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec0ad090 msgr2=0x7f31ec157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec0ad090 0x7f31ec157a90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009910 tx=0x7f31e40091a0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- 172.21.15.146:0/3400488740 shutdown_connections 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec0ad090 0x7f31ec157a90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/3400488740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- 172.21.15.146:0/3400488740 >> 172.21.15.146:0/3400488740 conn(0x7f31ec06e070 msgr2=0x7f31ec0b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- 172.21.15.146:0/3400488740 shutdown_connections 2024-08-15T17:00:44.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- 172.21.15.146:0/3400488740 wait complete. 2024-08-15T17:00:44.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 Processor -- start 2024-08-15T17:00:44.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.616+0000 7f31f1cd5700 1 -- start start 2024-08-15T17:00:44.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 0x7f31ec1468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec1670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1680d0 con 0x7f31ec155d80 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0b0380 con 0x7f31ec0ad090 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec146df0 con 0x7f31ec0af5f0 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34096/0 (socket says 172.21.15.146:34096) 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 -- 172.21.15.146:0/4168305946 learned_addr learned my addr 172.21.15.146:0/4168305946 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31ebfff700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec1670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f0cd3700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 0x7f31ec1468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 -- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 msgr2=0x7f31ec1670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec1670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 -- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 msgr2=0x7f31ec1468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 0x7f31ec1468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 -- 172.21.15.146:0/4168305946 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4009420 con 0x7f31ec155d80 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31ebfff700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f0cd3700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 0x7f31ec1468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f14d4700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e400a010 tx=0x7f31e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e401bdb0 con 0x7f31ec155d80 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e401c450 con 0x7f31ec155d80 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec167b10 con 0x7f31ec155d80 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e401c810 con 0x7f31ec155d80 2024-08-15T17:00:44.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec0b0010 con 0x7f31ec155d80 2024-08-15T17:00:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.620+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec0afe70 con 0x7f31ec155d80 2024-08-15T17:00:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.624+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f31e401cc00 con 0x7f31ec155d80 2024-08-15T17:00:44.635 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.624+0000 7f31e8ff9700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 0x7f31cc062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.624+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f31e407e6d0 con 0x7f31ec155d80 2024-08-15T17:00:44.636 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.624+0000 7f31f0cd3700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 0x7f31cc062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.637 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.624+0000 7f31f0cd3700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 0x7f31cc062d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31ec1555b0 tx=0x7f31d4005c50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.632+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f31ec0afe70 con 0x7f31ec155d80 2024-08-15T17:00:44.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.828+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000940580", "format": "json"} v 0) v1 -- 0x7f31ec0afe70 con 0x7f31ec155d80 2024-08-15T17:00:44.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.828+0000 7f31e8ff9700 1 -- 172.21.15.146:0/4168305946 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000940580", "format": "json"}]=0 v16) v1 ==== 110+0+1846 (secure 0 0 0) 0x7f31ec0afe70 con 0x7f31ec155d80 2024-08-15T17:00:44.842 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:00:44.842 INFO:teuthology.orchestra.run.smithi146.stdout:{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:43.830602+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":0},"id":3} 2024-08-15T17:00:44.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.832+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 msgr2=0x7f31cc062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.832+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 0x7f31cc062d40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f31ec1555b0 tx=0x7f31d4005c50 comp rx=0 tx=0).stop 2024-08-15T17:00:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.832+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 msgr2=0x7f31ec1485b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.832+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e400a010 tx=0x7f31e4016000 comp rx=0 tx=0).stop 2024-08-15T17:00:44.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 shutdown_connections 2024-08-15T17:00:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f31cc060890 0x7f31cc062d40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f31ec155d80 0x7f31ec1485b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0af5f0 0x7f31ec1670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 --2- 172.21.15.146:0/4168305946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec0ad090 0x7f31ec1468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.836+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 >> 172.21.15.146:0/4168305946 conn(0x7f31ec06e070 msgr2=0x7f31ec158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.840+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 shutdown_connections 2024-08-15T17:00:44.853 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.840+0000 7f31f1cd5700 1 -- 172.21.15.146:0/4168305946 wait complete. 2024-08-15T17:00:44.871 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000940580 --yes-i-really-mean-it 2024-08-15T17:00:44.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.960+0000 7f555b294700 1 Processor -- start 2024-08-15T17:00:44.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.960+0000 7f555b294700 1 -- start start 2024-08-15T17:00:44.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f555413fa50 0x7f55540aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f555406e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540b0810 con 0x7f555413f680 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55540afc70 con 0x7f555413fa50 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540af060 con 0x7f55540ad6c0 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f555413fa50 0x7f55540aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f555413fa50 0x7f55540aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34112/0 (socket says 172.21.15.146:34112) 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 -- 172.21.15.146:0/2393696912 learned_addr learned my addr 172.21.15.146:0/2393696912 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555aa93700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f555406e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 -- 172.21.15.146:0/2393696912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 msgr2=0x7f555406e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f555406e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 -- 172.21.15.146:0/2393696912 >> v1:172.21.15.146:6789/0 conn(0x7f555413f680 legacy=0x7f55540b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 -- 172.21.15.146:0/2393696912 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540af340 con 0x7f555413fa50 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555aa93700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f555406e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:00:44.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559a91700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413fa50 0x7f55540aea80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55440099b0 tx=0x7f5544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0759a2e0fe7d3ac server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559290700 1 -- 172.21.15.146:0/2393696912 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400b860 con 0x7f555413fa50 2024-08-15T17:00:44.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559290700 1 -- 172.21.15.146:0/2393696912 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554400b9c0 con 0x7f555413fa50 2024-08-15T17:00:44.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f5559290700 1 -- 172.21.15.146:0/2393696912 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400bd60 con 0x7f555413fa50 2024-08-15T17:00:44.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- 172.21.15.146:0/2393696912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413fa50 msgr2=0x7f55540aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413fa50 0x7f55540aea80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55440099b0 tx=0x7f5544009250 comp rx=0 tx=0).stop 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- 172.21.15.146:0/2393696912 shutdown_connections 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f555406e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 --2- 172.21.15.146:0/2393696912 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413fa50 0x7f55540aea80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- 172.21.15.146:0/2393696912 >> 172.21.15.146:0/2393696912 conn(0x7f55540678a0 msgr2=0x7f5554067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- 172.21.15.146:0/2393696912 shutdown_connections 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.964+0000 7f555b294700 1 -- 172.21.15.146:0/2393696912 wait complete. 2024-08-15T17:00:44.979 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 Processor -- start 2024-08-15T17:00:44.980 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- start start 2024-08-15T17:00:44.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f5554153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413f680 0x7f55540a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555413fa50 0x7f55540a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540a9410 con 0x7f555413f680 2024-08-15T17:00:44.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555416a160 con 0x7f555413fa50 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5554154250 con 0x7f55540ad6c0 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413f680 0x7f55540a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413f680 0x7f55540a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34122/0 (socket says 172.21.15.146:34122) 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 -- 172.21.15.146:0/252023432 learned_addr learned my addr 172.21.15.146:0/252023432 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555a292700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f5554153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555aa93700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555413fa50 0x7f55540a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 -- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 msgr2=0x7f5554153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55540ad6c0 0x7f5554153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 -- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555413fa50 msgr2=0x7f55540a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f555413fa50 0x7f55540a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:00:44.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 -- 172.21.15.146:0/252023432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440094f0 con 0x7f555413f680 2024-08-15T17:00:44.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f5559a91700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f555413f680 0x7f55540a88e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f555406fbb0 tx=0x7f5544017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544024440 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55440245a0 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- 172.21.15.146:0/252023432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555416a5f0 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544024960 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.968+0000 7f555b294700 1 -- 172.21.15.146:0/252023432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5554169b10 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f555b294700 1 -- 172.21.15.146:0/252023432 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555416a310 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f554401a420 con 0x7f555413f680 2024-08-15T17:00:44.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f554a7fc700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5538060890 0x7f5538062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:00:44.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f554407ed00 con 0x7f555413f680 2024-08-15T17:00:44.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f555a292700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5538060890 0x7f5538062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:00:44.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.972+0000 7f555a292700 1 --2- 172.21.15.146:0/252023432 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5538060890 0x7f5538062d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f555416cbf0 tx=0x7f5550009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:00:44.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:44.976+0000 7f554a7fc700 1 -- 172.21.15.146:0/252023432 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f555416a310 con 0x7f555413f680 2024-08-15T17:00:45.175 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:00:45.164+0000 7f555b294700 1 -- 172.21.15.146:0/252023432 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000940580", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f555416a2d0 con 0x7f5538060890 2024-08-15T17:00:47.106 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:00:47.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:01:17.158 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:01:17.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:01:47.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:01:47.220 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:02:17.265 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:02:17.268 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:02:44.887 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-08-15T17:02:44.888 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-15T17:02:44.889 DEBUG:teuthology.orchestra.run.smithi146:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:02:44.908 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-15T17:02:44.930 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-15T17:02:44.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi146.front.sepia.ceph.com... 2024-08-15T17:02:44.930 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:02:44.931 DEBUG:teuthology.orchestra.run.smithi146:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-15T17:02:44.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi146.stderr:ceph-fuse[33510]: fuse finished with error 0 and tester_r 0 2024-08-15T17:02:44.967 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:02:44.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:44.967 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-15T17:02:45.003 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi161.front.sepia.ceph.com... 2024-08-15T17:02:45.003 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-15T17:02:45.003 DEBUG:teuthology.orchestra.run.smithi161:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-08-15T17:02:45.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi161.stderr:ceph-fuse[29427]: fuse finished with error 0 and tester_r 0 2024-08-15T17:02:45.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:02:45.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:45.021 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-15T17:02:45.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:02:45.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:45.141 DEBUG:teuthology.orchestra.run.smithi146:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-15T17:02:45.148 INFO:teuthology.orchestra.run.smithi146.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-08-15T17:02:45.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:02:45.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:02:45.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:45.150 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-15T17:02:45.156 INFO:teuthology.orchestra.run.smithi161.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-08-15T17:02:45.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:02:45.158 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:45.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 Processor -- start 2024-08-15T17:02:45.304 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- start start 2024-08-15T17:02:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fedbc155e50 0x7fedbc0a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fedbc06f2f0 0x7fedbc0aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedbc0afc10 con 0x7fedbc156220 2024-08-15T17:02:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedbc0b10b0 con 0x7fedbc155e50 2024-08-15T17:02:45.305 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedbc0aed20 con 0x7fedbc06f2f0 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fedbc155e50 0x7fedbc0a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedb37fe700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fedbc156220 0x7fedbc06ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60426/0 (socket says 172.21.15.146:60426) 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedb37fe700 1 -- 172.21.15.146:0/4216934774 learned_addr learned my addr 172.21.15.146:0/4216934774 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc0d26700 1 --2- 172.21.15.146:0/4216934774 >> v2:172.21.15.161:3300/0 conn(0x7fedbc06f2f0 0x7fedbc0aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbb7fe700 1 -- 172.21.15.146:0/4216934774 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716179500 0 0) 0x7fedbc0afc10 con 0x7fedbc156220 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbb7fe700 1 -- 172.21.15.146:0/4216934774 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda8003570 con 0x7fedbc156220 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 -- 172.21.15.146:0/4216934774 >> v2:172.21.15.161:3300/0 conn(0x7fedbc06f2f0 msgr2=0x7fedbc0aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 --2- 172.21.15.146:0/4216934774 >> v2:172.21.15.161:3300/0 conn(0x7fedbc06f2f0 0x7fedbc0aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 -- 172.21.15.146:0/4216934774 >> v1:172.21.15.146:6789/0 conn(0x7fedbc156220 legacy=0x7fedbc06ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.306 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 -- 172.21.15.146:0/4216934774 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc0af000 con 0x7fedbc155e50 2024-08-15T17:02:45.307 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbbfff700 1 --2- 172.21.15.146:0/4216934774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc155e50 0x7fedbc0a9ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0aeab0 tx=0x7fedac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c1a1682b9332c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbb7fe700 1 -- 172.21.15.146:0/4216934774 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011630 con 0x7fedbc155e50 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbb7fe700 1 -- 172.21.15.146:0/4216934774 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac0117b0 con 0x7fedbc155e50 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedbb7fe700 1 -- 172.21.15.146:0/4216934774 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011b70 con 0x7fedbc155e50 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- 172.21.15.146:0/4216934774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc155e50 msgr2=0x7fedbc0a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 --2- 172.21.15.146:0/4216934774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc155e50 0x7fedbc0a9ee0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fedbc0aeab0 tx=0x7fedac0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- 172.21.15.146:0/4216934774 shutdown_connections 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 --2- 172.21.15.146:0/4216934774 >> v2:172.21.15.161:3300/0 conn(0x7fedbc06f2f0 0x7fedbc0aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 --2- 172.21.15.146:0/4216934774 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc155e50 0x7fedbc0a9ee0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- 172.21.15.146:0/4216934774 >> 172.21.15.146:0/4216934774 conn(0x7fedbc066e00 msgr2=0x7fedbc067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- 172.21.15.146:0/4216934774 shutdown_connections 2024-08-15T17:02:45.308 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.290+0000 7fedc2789700 1 -- 172.21.15.146:0/4216934774 wait complete. 2024-08-15T17:02:45.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 Processor -- start 2024-08-15T17:02:45.309 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 -- start start 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedbc155e50 0x7fedbc0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedbc156220 0x7fedbc06a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc098950 con 0x7fedbc06f2f0 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc098c30 con 0x7fedbc156220 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc2789700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc098f10 con 0x7fedbc155e50 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46026/0 (socket says 172.21.15.146:46026) 2024-08-15T17:02:45.311 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 -- 172.21.15.146:0/2683559560 learned_addr learned my addr 172.21.15.146:0/2683559560 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedb37fe700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedbc155e50 0x7fedbc0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedc0d26700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedbc156220 0x7fedbc06a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 -- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedbc155e50 msgr2=0x7fedbc0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedbc155e50 0x7fedbc0cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 -- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedbc156220 msgr2=0x7fedbc06a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedbc156220 0x7fedbc06a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.312 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.294+0000 7fedbbfff700 1 -- 172.21.15.146:0/2683559560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac009420 con 0x7fedbc06f2f0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedbbfff700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fedac013000 tx=0x7fedac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac01f070 con 0x7fedbc06f2f0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac000bc0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc0991f0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011b50 con 0x7fedbc06f2f0 2024-08-15T17:02:45.313 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc13ad30 con 0x7fedbc06f2f0 2024-08-15T17:02:45.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc0645c0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fedac023070 con 0x7fedbc06f2f0 2024-08-15T17:02:45.314 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 0x7fed9c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.298+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fedac07e1d0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.302+0000 7fedb37fe700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 0x7fed9c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.302+0000 7fedb37fe700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 0x7fed9c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedbc160e90 tx=0x7feda4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.322 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.306+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fedac04e2a0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.506+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedbc0ae670 con 0x7fedbc06f2f0 2024-08-15T17:02:45.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.506+0000 7fedb8ff9700 1 -- 172.21.15.146:0/2683559560 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5692 (secure 0 0 0) 0x7fedac0518c0 con 0x7fedbc06f2f0 2024-08-15T17:02:45.522 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:45.522 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 msgr2=0x7fed9c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 0x7fed9c062d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fedbc160e90 tx=0x7feda4009040 comp rx=0 tx=0).stop 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 msgr2=0x7fedbc0999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fedac013000 tx=0x7fedac017000 comp rx=0 tx=0).stop 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 shutdown_connections 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fed9c060860 0x7fed9c062d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fedbc156220 0x7fedbc06a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fedbc155e50 0x7fedbc0cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.525 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 --2- 172.21.15.146:0/2683559560 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fedbc06f2f0 0x7fedbc0999f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 >> 172.21.15.146:0/2683559560 conn(0x7fedbc066e00 msgr2=0x7fedbc151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 shutdown_connections 2024-08-15T17:02:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.510+0000 7fedc2789700 1 -- 172.21.15.146:0/2683559560 wait complete. 2024-08-15T17:02:45.526 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-08-15T17:02:45.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:45.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.626+0000 7f190aef9700 1 Processor -- start 2024-08-15T17:02:45.643 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.626+0000 7f190aef9700 1 -- start start 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f19040cb560 0x7f1904161ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19040cbaa0 con 0x7f1904154900 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1904157630 con 0x7f19040b14a0 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1904068790 con 0x7f19040cb560 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f19040cb560 0x7f1904161ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f19040cb560 0x7f1904161ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46034/0 (socket says 172.21.15.146:46034) 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 -- 172.21.15.146:0/3624204662 learned_addr learned my addr 172.21.15.146:0/3624204662 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 -- 172.21.15.146:0/3624204662 >> v1:172.21.15.161:6789/0 conn(0x7f19040b14a0 legacy=0x7f19040cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 -- 172.21.15.146:0/3624204662 >> v1:172.21.15.146:6789/0 conn(0x7f1904154900 legacy=0x7f19040b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.644 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 -- 172.21.15.146:0/3624204662 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904157910 con 0x7f19040cb560 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190a6f8700 1 --2- 172.21.15.146:0/3624204662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040cb560 0x7f1904161ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0099b0 tx=0x7f18fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61281f3c9d96a8e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f1908ef5700 1 -- 172.21.15.146:0/3624204662 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00b860 con 0x7f19040cb560 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f1908ef5700 1 -- 172.21.15.146:0/3624204662 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00b9c0 con 0x7f19040cb560 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f1908ef5700 1 -- 172.21.15.146:0/3624204662 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00bd60 con 0x7f19040cb560 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- 172.21.15.146:0/3624204662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040cb560 msgr2=0x7f1904161ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 --2- 172.21.15.146:0/3624204662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040cb560 0x7f1904161ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0099b0 tx=0x7f18fc009250 comp rx=0 tx=0).stop 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- 172.21.15.146:0/3624204662 shutdown_connections 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 --2- 172.21.15.146:0/3624204662 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040cb560 0x7f1904161ad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- 172.21.15.146:0/3624204662 >> 172.21.15.146:0/3624204662 conn(0x7f19040678d0 msgr2=0x7f1904067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:45.646 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- 172.21.15.146:0/3624204662 shutdown_connections 2024-08-15T17:02:45.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.630+0000 7f190aef9700 1 -- 172.21.15.146:0/3624204662 wait complete. 2024-08-15T17:02:45.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 Processor -- start 2024-08-15T17:02:45.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 -- start start 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19040cb560 0x7f190415a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 0x7f1904152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190415b160 con 0x7f19040b14a0 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190416b030 con 0x7f19040cb560 2024-08-15T17:02:45.651 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190aef9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190415ae80 con 0x7f1904154900 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46038/0 (socket says 172.21.15.146:46038) 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 -- 172.21.15.146:0/1566814591 learned_addr learned my addr 172.21.15.146:0/1566814591 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190a6f8700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 0x7f1904152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f19096f6700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19040cb560 0x7f190415a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 -- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 msgr2=0x7f1904152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 0x7f1904152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 -- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19040cb560 msgr2=0x7f190415a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19040cb560 0x7f190415a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 -- 172.21.15.146:0/1566814591 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc0094f0 con 0x7f19040b14a0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f190a6f8700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 0x7f1904152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f1909ef7700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18f400c4e0 tx=0x7f18f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4011590 con 0x7f19040b14a0 2024-08-15T17:02:45.652 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f4011710 con 0x7f19040b14a0 2024-08-15T17:02:45.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.634+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4011ad0 con 0x7f19040b14a0 2024-08-15T17:02:45.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190416a9e0 con 0x7f19040b14a0 2024-08-15T17:02:45.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190414e660 con 0x7f19040b14a0 2024-08-15T17:02:45.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19040643a0 con 0x7f19040b14a0 2024-08-15T17:02:45.653 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f18f401b3b0 con 0x7f19040b14a0 2024-08-15T17:02:45.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f18f9ffb700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 0x7f18f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f19096f6700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 0x7f18f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.654 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.638+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f18f407c560 con 0x7f19040b14a0 2024-08-15T17:02:45.655 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.642+0000 7f19096f6700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 0x7f18f0062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f190416a7a0 tx=0x7f1900009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.650+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f18f404c590 con 0x7f19040b14a0 2024-08-15T17:02:45.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.818+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f190406f5f0 con 0x7f19040b14a0 2024-08-15T17:02:45.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.818+0000 7f18f9ffb700 1 -- 172.21.15.146:0/1566814591 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18647 (secure 0 0 0) 0x7f18f404fbb0 con 0x7f19040b14a0 2024-08-15T17:02:45.834 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:45.835 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:45.834731+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 msgr2=0x7f18f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 0x7f18f0062d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f190416a7a0 tx=0x7f1900009040 comp rx=0 tx=0).stop 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 msgr2=0x7f190414e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f18f400c4e0 tx=0x7f18f400c8a0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 shutdown_connections 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f18f0060860 0x7f18f0062d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1904154900 0x7f1904152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19040cb560 0x7f190415a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 --2- 172.21.15.146:0/1566814591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f19040b14a0 0x7f190414e120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 >> 172.21.15.146:0/1566814591 conn(0x7f19040678d0 msgr2=0x7f1904151950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 shutdown_connections 2024-08-15T17:02:45.838 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.822+0000 7f190aef9700 1 -- 172.21.15.146:0/1566814591 wait complete. 2024-08-15T17:02:45.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-15T17:00:27.040042+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-08-15T17:00:27.765439+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000940580.meta', 'create_time': '2024-08-15T17:00:41.772696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000940580'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000940580.data', 'create_time': '2024-08-15T17:00:41.868946+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000940580'}}}] 2024-08-15T17:02:45.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:02:45.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.946+0000 7fcb33131700 1 Processor -- start 2024-08-15T17:02:45.963 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 -- start start 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb2c1554c0 0x7fcb2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcb2c068db0 0x7fcb2c06b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c06b000 con 0x7fcb2c069ee0 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c06a3f0 con 0x7fcb2c1554c0 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb33131700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c06a6d0 con 0x7fcb2c068db0 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb2c1554c0 0x7fcb2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fcb2c1554c0 0x7fcb2c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46048/0 (socket says 172.21.15.146:46048) 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 -- 172.21.15.146:0/483509464 learned_addr learned my addr 172.21.15.146:0/483509464 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb316ce700 1 --2- 172.21.15.146:0/483509464 >> v2:172.21.15.161:3300/0 conn(0x7fcb2c068db0 0x7fcb2c06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.968 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 -- 172.21.15.146:0/483509464 >> v2:172.21.15.161:3300/0 conn(0x7fcb2c068db0 msgr2=0x7fcb2c06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 --2- 172.21.15.146:0/483509464 >> v2:172.21.15.161:3300/0 conn(0x7fcb2c068db0 0x7fcb2c06b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 -- 172.21.15.146:0/483509464 >> v1:172.21.15.161:6789/0 conn(0x7fcb2c069ee0 legacy=0x7fcb2c0686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 -- 172.21.15.146:0/483509464 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c06a9b0 con 0x7fcb2c1554c0 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb30ecd700 1 --2- 172.21.15.146:0/483509464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c1554c0 0x7fcb2c0699a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c06d980 tx=0x7fcb20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a736e2d16c71f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb2b7fe700 1 -- 172.21.15.146:0/483509464 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000b860 con 0x7fcb2c1554c0 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb2b7fe700 1 -- 172.21.15.146:0/483509464 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000b9c0 con 0x7fcb2c1554c0 2024-08-15T17:02:45.969 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.950+0000 7fcb2b7fe700 1 -- 172.21.15.146:0/483509464 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000bd60 con 0x7fcb2c1554c0 2024-08-15T17:02:45.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- 172.21.15.146:0/483509464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c1554c0 msgr2=0x7fcb2c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- 172.21.15.146:0/483509464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c1554c0 0x7fcb2c0699a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c06d980 tx=0x7fcb20009250 comp rx=0 tx=0).stop 2024-08-15T17:02:45.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- 172.21.15.146:0/483509464 shutdown_connections 2024-08-15T17:02:45.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- 172.21.15.146:0/483509464 >> v2:172.21.15.161:3300/0 conn(0x7fcb2c068db0 0x7fcb2c06b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- 172.21.15.146:0/483509464 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c1554c0 0x7fcb2c0699a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- 172.21.15.146:0/483509464 >> 172.21.15.146:0/483509464 conn(0x7fcb2c0c9bf0 msgr2=0x7fcb2c06d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:45.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- 172.21.15.146:0/483509464 shutdown_connections 2024-08-15T17:02:45.971 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- 172.21.15.146:0/483509464 wait complete. 2024-08-15T17:02:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 Processor -- start 2024-08-15T17:02:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- start start 2024-08-15T17:02:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c068db0 0x7fcb2c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c1554c0 0x7fcb2c06b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c16acb0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0c5d10 con 0x7fcb2c068db0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb33131700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0c5ff0 con 0x7fcb2c1554c0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb30ecd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c068db0 0x7fcb2c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46058/0 (socket says 172.21.15.146:46058) 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb2bfff700 1 -- 172.21.15.146:0/4286351571 learned_addr learned my addr 172.21.15.146:0/4286351571 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.954+0000 7fcb316ce700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c1554c0 0x7fcb2c06b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 -- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c1554c0 msgr2=0x7fcb2c06b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c1554c0 0x7fcb2c06b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 -- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c068db0 msgr2=0x7fcb2c14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c068db0 0x7fcb2c14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:45.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 -- 172.21.15.146:0/4286351571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200094f0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb2bfff700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800c590 tx=0x7fcb1800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18011590 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb18011710 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c1564d0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18011ad0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c156a90 con 0x7fcb2c069ee0 2024-08-15T17:02:45.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c0645c0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fcb1801b3b0 con 0x7fcb2c069ee0 2024-08-15T17:02:45.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.958+0000 7fcb28ff9700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 0x7fcb10062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:45.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.962+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fcb1807c320 con 0x7fcb2c069ee0 2024-08-15T17:02:45.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.962+0000 7fcb30ecd700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 0x7fcb10062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:45.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.962+0000 7fcb30ecd700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 0x7fcb10062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c06d980 tx=0x7fcb20017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:45.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:45.966+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fcb1804c3f0 con 0x7fcb2c069ee0 2024-08-15T17:02:46.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.126+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb2c0c7e30 con 0x7fcb2c069ee0 2024-08-15T17:02:46.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.126+0000 7fcb28ff9700 1 -- 172.21.15.146:0/4286351571 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb1804fa10 con 0x7fcb2c069ee0 2024-08-15T17:02:46.143 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 msgr2=0x7fcb10062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 0x7fcb10062d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c06d980 tx=0x7fcb20017000 comp rx=0 tx=0).stop 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 msgr2=0x7fcb2c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800c590 tx=0x7fcb1800c8a0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 shutdown_connections 2024-08-15T17:02:46.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fcb10060860 0x7fcb10062d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb2c1554c0 0x7fcb2c06b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcb2c069ee0 0x7fcb2c155aa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 --2- 172.21.15.146:0/4286351571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb2c068db0 0x7fcb2c14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 >> 172.21.15.146:0/4286351571 conn(0x7fcb2c0c9bf0 msgr2=0x7fcb2c0c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 shutdown_connections 2024-08-15T17:02:46.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.130+0000 7fcb33131700 1 -- 172.21.15.146:0/4286351571 wait complete. 2024-08-15T17:02:46.157 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-15T17:02:46.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 Processor -- start 2024-08-15T17:02:46.269 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 -- start start 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 0x7faaa806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 0x7faaa8152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa80ace30 con 0x7faaa80b1830 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80ae6b0 con 0x7faaa806ead0 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaace8b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8156dd0 con 0x7faaa8152bd0 2024-08-15T17:02:46.271 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa6ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7faaa80b1830 0x7faaa8155c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60440/0 (socket says 172.21.15.146:60440) 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa6ffd700 1 -- 172.21.15.146:0/3326301812 learned_addr learned my addr 172.21.15.146:0/3326301812 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa7fff700 1 --2- 172.21.15.146:0/3326301812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 0x7faaa8152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa77fe700 1 --2- 172.21.15.146:0/3326301812 >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 0x7faaa806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143839590 0 0) 0x7faaa80ace30 con 0x7faaa80b1830 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa88003570 con 0x7faaa80b1830 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa7fff700 1 -- 172.21.15.146:0/3326301812 >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 msgr2=0x7faaa806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa7fff700 1 --2- 172.21.15.146:0/3326301812 >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 0x7faaa806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa7fff700 1 -- 172.21.15.146:0/3326301812 >> v1:172.21.15.146:6789/0 conn(0x7faaa80b1830 legacy=0x7faaa8155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa7fff700 1 -- 172.21.15.146:0/3326301812 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8156450 con 0x7faaa8152bd0 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693338583 0 0) 0x7faa88003570 con 0x7faaa80b1830 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.254+0000 7faaa77fe700 1 --2- 172.21.15.146:0/3326301812 >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 0x7faaa806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:46.272 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaa7fff700 1 --2- 172.21.15.146:0/3326301812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 0x7faaa8152fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faaa00098e0 tx=0x7faaa00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfa16f33a0c5dfc server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0011630 con 0x7faaa8152bd0 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0011790 con 0x7faaa8152bd0 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaa67fc700 1 -- 172.21.15.146:0/3326301812 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0011b30 con 0x7faaa8152bd0 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- 172.21.15.146:0/3326301812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 msgr2=0x7faaa8152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 --2- 172.21.15.146:0/3326301812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 0x7faaa8152fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faaa00098e0 tx=0x7faaa00091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- 172.21.15.146:0/3326301812 shutdown_connections 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 --2- 172.21.15.146:0/3326301812 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa8152bd0 0x7faaa8152fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.274 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 --2- 172.21.15.146:0/3326301812 >> v2:172.21.15.161:3300/0 conn(0x7faaa806ead0 0x7faaa806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- 172.21.15.146:0/3326301812 >> 172.21.15.146:0/3326301812 conn(0x7faaa80678a0 msgr2=0x7faaa8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- 172.21.15.146:0/3326301812 shutdown_connections 2024-08-15T17:02:46.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- 172.21.15.146:0/3326301812 wait complete. 2024-08-15T17:02:46.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 Processor -- start 2024-08-15T17:02:46.275 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.258+0000 7faaace8b700 1 -- start start 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa806ead0 0x7faaa814b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa80b1830 0x7faaa814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8152fa0 con 0x7faaa8152bd0 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80ac330 con 0x7faaa806ead0 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81531f0 con 0x7faaa80b1830 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46066/0 (socket says 172.21.15.146:46066) 2024-08-15T17:02:46.277 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 -- 172.21.15.146:0/1404576447 learned_addr learned my addr 172.21.15.146:0/1404576447 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa77fe700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa806ead0 0x7faaa814b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa6ffd700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa80b1830 0x7faaa814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 -- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa80b1830 msgr2=0x7faaa814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa80b1830 0x7faaa814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 -- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa806ead0 msgr2=0x7faaa814b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa806ead0 0x7faaa814b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 -- 172.21.15.146:0/1404576447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009420 con 0x7faaa8152bd0 2024-08-15T17:02:46.278 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaa7fff700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faaa8156f10 tx=0x7faaa0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa001f070 con 0x7faaa8152bd0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa0002f70 con 0x7faaa8152bd0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa0009420 con 0x7faaa8152bd0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0025420 con 0x7faaa8152bd0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.262+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa81537b0 con 0x7faaa8152bd0 2024-08-15T17:02:46.279 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa81539d0 con 0x7faaa8152bd0 2024-08-15T17:02:46.280 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7faaa0023070 con 0x7faaa8152bd0 2024-08-15T17:02:46.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faa9f7fe700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 0x7faa8c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7faaa007e3e0 con 0x7faaa8152bd0 2024-08-15T17:02:46.281 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faaa77fe700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 0x7faa8c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.282 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.266+0000 7faaa77fe700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 0x7faa8c062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ea0 tx=0x7faa900091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.274+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7faaa004e4d0 con 0x7faaa8152bd0 2024-08-15T17:02:46.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faaa8064330 con 0x7faaa8152bd0 2024-08-15T17:02:46.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faa9f7fe700 1 -- 172.21.15.146:0/1404576447 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7faaa0051af0 con 0x7faaa8152bd0 2024-08-15T17:02:46.453 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:02:46.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 msgr2=0x7faa8c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 0x7faa8c062d10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa90009ea0 tx=0x7faa900091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 msgr2=0x7faaa8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7faaa8156f10 tx=0x7faaa0017000 comp rx=0 tx=0).stop 2024-08-15T17:02:46.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 shutdown_connections 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7faa8c060860 0x7faa8c062d10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7faaa8152bd0 0x7faaa8156fd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaa80b1830 0x7faaa814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 --2- 172.21.15.146:0/1404576447 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaa806ead0 0x7faaa814b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.438+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 >> 172.21.15.146:0/1404576447 conn(0x7faaa80678a0 msgr2=0x7faaa806a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.442+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 shutdown_connections 2024-08-15T17:02:46.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.442+0000 7faaace8b700 1 -- 172.21.15.146:0/1404576447 wait complete. 2024-08-15T17:02:46.467 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-15T17:02:46.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 Processor -- start 2024-08-15T17:02:46.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 -- start start 2024-08-15T17:02:46.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f493c155760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c156a70 con 0x7f493c06b0f0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f493c155e60 con 0x7f493c156f00 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0b1b20 con 0x7f493c0b16d0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f493c06b0f0 0x7f493c13e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:60442/0 (socket says 172.21.15.146:60442) 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493a59c700 1 -- 172.21.15.146:0/3470034650 learned_addr learned my addr 172.21.15.146:0/3470034650 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f4939d9b700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f493c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752999317 0 0) 0x7f493c156a70 con 0x7f493c06b0f0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4928003570 con 0x7f493c06b0f0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720374914 0 0) 0x7f493c155e60 con 0x7f493c156f00 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c156a70 con 0x7f493c156f00 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3989376500 0 0) 0x7f4928003570 con 0x7f493c06b0f0 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 msgr2=0x7f493c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f493c155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.581 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 >> v1:172.21.15.161:6789/0 conn(0x7f493c156f00 legacy=0x7f493c157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.562+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c156140 con 0x7f493c06b0f0 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f4934001470 con 0x7f493c06b0f0 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 >> v1:172.21.15.146:6789/0 conn(0x7f493c06b0f0 legacy=0x7f493c13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f4928003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4928004180 0x7f4928004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4928004d50 0x7f49280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0b1b20 con 0x7f4928004d50 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492800bba0 con 0x7f4928004180 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492800bd60 con 0x7f493c0b16d0 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4928004d50 0x7f49280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493a59c700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f4928003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f4939d9b700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4928004180 0x7f4928004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 -- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 msgr2=0x7f4928003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f493c0b16d0 0x7f4928003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 -- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4928004180 msgr2=0x7f4928004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4928004180 0x7f4928004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 -- 172.21.15.146:0/3470034650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928003570 con 0x7f4928004d50 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4928004d50 0x7f49280051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f492c002120 tx=0x7f492c0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afa7e143aedaf39 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c001ab0 con 0x7f4928004d50 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c00c980 con 0x7f4928004d50 2024-08-15T17:02:46.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493959a700 1 -- 172.21.15.146:0/3470034650 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00cd20 con 0x7f4928004d50 2024-08-15T17:02:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 -- 172.21.15.146:0/3470034650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4928004d50 msgr2=0x7f49280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 --2- 172.21.15.146:0/3470034650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4928004d50 0x7f49280051c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f492c002120 tx=0x7f492c0017a0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493ad9d700 1 -- 172.21.15.146:0/3470034650 reap_dead start 2024-08-15T17:02:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 -- 172.21.15.146:0/3470034650 shutdown_connections 2024-08-15T17:02:46.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 -- 172.21.15.146:0/3470034650 >> 172.21.15.146:0/3470034650 conn(0x7f493c0678a0 msgr2=0x7f493c067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 -- 172.21.15.146:0/3470034650 shutdown_connections 2024-08-15T17:02:46.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.566+0000 7f493b59e700 1 -- 172.21.15.146:0/3470034650 wait complete. 2024-08-15T17:02:46.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 Processor -- start 2024-08-15T17:02:46.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- start start 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 0x7f493c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f493c13d940 0x7f493c0a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c13cbb0 con 0x7f493c13ddb0 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493c13bd40 con 0x7f493c13d940 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493c0a6230 con 0x7f4928004d50 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46084/0 (socket says 172.21.15.146:46084) 2024-08-15T17:02:46.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 -- 172.21.15.146:0/1018687629 learned_addr learned my addr 172.21.15.146:0/1018687629 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493a59c700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 0x7f493c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f4939d9b700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f493c13d940 0x7f493c0a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 -- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 msgr2=0x7f493c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 0x7f493c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 -- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f493c13d940 msgr2=0x7f493c0a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f493c13d940 0x7f493c0a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 -- 172.21.15.146:0/1018687629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c13bfa0 con 0x7f493c13ddb0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493a59c700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 0x7f493c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493ad9d700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f492c003000 tx=0x7f492c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c00c6d0 con 0x7f493c13ddb0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492c006db0 con 0x7f493c13ddb0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493c13c280 con 0x7f493c13ddb0 2024-08-15T17:02:46.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c012cf0 con 0x7f493c13ddb0 2024-08-15T17:02:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.570+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f493c13c840 con 0x7f493c13ddb0 2024-08-15T17:02:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f493c064330 con 0x7f493c13ddb0 2024-08-15T17:02:46.589 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f492c003030 con 0x7f493c13ddb0 2024-08-15T17:02:46.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f49327fc700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 0x7f491c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f492c07f230 con 0x7f493c13ddb0 2024-08-15T17:02:46.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f493a59c700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 0x7f491c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.590 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.574+0000 7f493a59c700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 0x7f491c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4934004a20 tx=0x7f4934001df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.582+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f492c04f260 con 0x7f493c13ddb0 2024-08-15T17:02:46.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.742+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f493c158ad0 con 0x7f493c13ddb0 2024-08-15T17:02:46.758 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.742+0000 7f49327fc700 1 -- 172.21.15.146:0/1018687629 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f492c052880 con 0x7f493c13ddb0 2024-08-15T17:02:46.759 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 64 2024-08-15T17:02:46.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 msgr2=0x7f491c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 0x7f491c062d10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4934004a20 tx=0x7f4934001df0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 msgr2=0x7f493c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f492c003000 tx=0x7f492c016000 comp rx=0 tx=0).stop 2024-08-15T17:02:46.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 shutdown_connections 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f491c060860 0x7f491c062d10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f493c13ddb0 0x7f493c0a5990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f493c13d940 0x7f493c0a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 --2- 172.21.15.146:0/1018687629 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4928004d50 0x7f493c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 >> 172.21.15.146:0/1018687629 conn(0x7f493c0678a0 msgr2=0x7f493c068f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 shutdown_connections 2024-08-15T17:02:46.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.746+0000 7f493b59e700 1 -- 172.21.15.146:0/1018687629 wait complete. 2024-08-15T17:02:46.773 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000940580.meta pg_num 2024-08-15T17:02:46.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 Processor -- start 2024-08-15T17:02:46.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 -- start start 2024-08-15T17:02:46.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 0x7f8c1c159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c1c066e00 0x7f8c1c067280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1c0cf520 con 0x7f8c1c0cf9b0 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1c0677c0 con 0x7f8c1c066e00 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c20ad7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0d0f40 con 0x7f8c1c1590a0 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 0x7f8c1c159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1bfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c1c066e00 0x7f8c1c067280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1bfff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c1c066e00 0x7f8c1c067280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:35888/0 (socket says 172.21.15.146:35888) 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1affd700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8c1c0cf9b0 0x7f8c1c0cb0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:36136/0 (socket says 172.21.15.146:36136) 2024-08-15T17:02:46.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1bfff700 1 -- 172.21.15.146:0/2274020684 learned_addr learned my addr 172.21.15.146:0/2274020684 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.870+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 0x7f8c1c159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:37230/0 (socket says 172.21.15.146:37230) 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1bfff700 1 -- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 msgr2=0x7f8c1c159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1bfff700 1 --2- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 0x7f8c1c159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1bfff700 1 -- 172.21.15.146:0/2274020684 >> v1:172.21.15.161:6789/0 conn(0x7f8c1c0cf9b0 legacy=0x7f8c1c0cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1bfff700 1 -- 172.21.15.146:0/2274020684 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0ce690 con 0x7f8c1c066e00 2024-08-15T17:02:46.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1bfff700 1 --2- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c066e00 0x7f8c1c067280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009f20 tx=0x7f8c14009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe406e16771d13c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1a7fc700 1 -- 172.21.15.146:0/2274020684 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400e070 con 0x7f8c1c066e00 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1a7fc700 1 -- 172.21.15.146:0/2274020684 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c1400b9d0 con 0x7f8c1c066e00 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c1a7fc700 1 -- 172.21.15.146:0/2274020684 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c140092e0 con 0x7f8c1c066e00 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 -- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c066e00 msgr2=0x7f8c1c067280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c066e00 0x7f8c1c067280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8c14009f20 tx=0x7f8c14009840 comp rx=0 tx=0).stop 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 -- 172.21.15.146:0/2274020684 shutdown_connections 2024-08-15T17:02:46.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c066e00 0x7f8c1c067280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/2274020684 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c1590a0 0x7f8c1c159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 -- 172.21.15.146:0/2274020684 >> 172.21.15.146:0/2274020684 conn(0x7f8c1c068c80 msgr2=0x7f8c1c0d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 -- 172.21.15.146:0/2274020684 shutdown_connections 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.874+0000 7f8c20ad7700 1 -- 172.21.15.146:0/2274020684 wait complete. 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 Processor -- start 2024-08-15T17:02:46.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- start start 2024-08-15T17:02:46.894 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c0cf9b0 0x7f8c1c063570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c063ab0 con 0x7f8c1c1590a0 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0c7f80 con 0x7f8c1c0cf9b0 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c0c8260 con 0x7f8c1c066e00 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:37236/0 (socket says 172.21.15.146:37236) 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46088/0 (socket says 172.21.15.146:46088) 2024-08-15T17:02:46.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 -- 172.21.15.146:0/3524671109 learned_addr learned my addr 172.21.15.146:0/3524671109 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1affd700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c0cf9b0 0x7f8c1c063570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 -- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c0cf9b0 msgr2=0x7f8c1c063570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c0cf9b0 0x7f8c1c063570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 -- 172.21.15.146:0/3524671109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 msgr2=0x7f8c1c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 -- 172.21.15.146:0/3524671109 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14009ae0 con 0x7f8c1c066e00 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1bfff700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c1b7fe700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e960 tx=0x7f8c100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10013680 con 0x7f8c1c066e00 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c100137e0 con 0x7f8c1c066e00 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c0c8540 con 0x7f8c1c066e00 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10013b80 con 0x7f8c1c066e00 2024-08-15T17:02:46.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.878+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c0c8b00 con 0x7f8c1c066e00 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c064330 con 0x7f8c1c066e00 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8c100123d0 con 0x7f8c1c066e00 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c037fe700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 0x7f8c08062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f8c1007d350 con 0x7f8c1c066e00 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c1affd700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 0x7f8c08062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:46.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.882+0000 7f8c1affd700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 0x7f8c08062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0ca980 tx=0x7f8c0c007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:46.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:46.890+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8c1004d420 con 0x7f8c1c066e00 2024-08-15T17:02:47.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.054+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.meta", "var": "pg_num"} v 0) v1 -- 0x7f8c1c067b80 con 0x7f8c1c066e00 2024-08-15T17:02:47.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.054+0000 7f8c037fe700 1 -- 172.21.15.146:0/3524671109 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.meta", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7f8c10050a40 con 0x7f8c1c066e00 2024-08-15T17:02:47.070 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:02:47.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 msgr2=0x7f8c08062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 0x7f8c08062d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c0ca980 tx=0x7f8c0c007eb0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 msgr2=0x7f8c1c152830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000e960 tx=0x7f8c100091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 shutdown_connections 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c08060860 0x7f8c08062d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c1c1590a0 0x7f8c1c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c1c0cf9b0 0x7f8c1c063570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 --2- 172.21.15.146:0/3524671109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c1c066e00 0x7f8c1c152830 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 >> 172.21.15.146:0/3524671109 conn(0x7f8c1c068c80 msgr2=0x7f8c1c1571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 shutdown_connections 2024-08-15T17:02:47.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.058+0000 7f8c20ad7700 1 -- 172.21.15.146:0/3524671109 wait complete. 2024-08-15T17:02:47.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000940580.data pg_num 2024-08-15T17:02:47.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 Processor -- start 2024-08-15T17:02:47.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 -- start start 2024-08-15T17:02:47.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec067030 0x7f6cec067400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6cec0cfb00 0x7f6cec0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6cec0cb5c0 0x7f6cec158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0cf670 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.196 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0cea60 con 0x7f6cec0cfb00 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.178+0000 7f6cf1063700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0ced40 con 0x7f6cec067030 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6cec0cb5c0 0x7f6cec158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f6cec0cb5c0 0x7f6cec158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46100/0 (socket says 172.21.15.146:46100) 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 -- 172.21.15.146:0/2452587975 learned_addr learned my addr 172.21.15.146:0/2452587975 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6ceb7fe700 1 --2- 172.21.15.146:0/2452587975 >> v2:172.21.15.161:3300/0 conn(0x7f6cec0cfb00 0x7f6cec0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cebfff700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec067030 0x7f6cec067400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 -- 172.21.15.146:0/2452587975 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec067030 msgr2=0x7f6cec067400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec067030 0x7f6cec067400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 -- 172.21.15.146:0/2452587975 >> v2:172.21.15.161:3300/0 conn(0x7f6cec0cfb00 msgr2=0x7f6cec0cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 --2- 172.21.15.146:0/2452587975 >> v2:172.21.15.161:3300/0 conn(0x7f6cec0cfb00 0x7f6cec0cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 -- 172.21.15.146:0/2452587975 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0cf020 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf0862700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec158cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6cec069590 tx=0x7f6ce4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34c26f4ebf5b8fd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6ceaffd700 1 -- 172.21.15.146:0/2452587975 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e070 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6ceaffd700 1 -- 172.21.15.146:0/2452587975 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce400ba50 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6ceaffd700 1 -- 172.21.15.146:0/2452587975 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40092e0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 -- 172.21.15.146:0/2452587975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 msgr2=0x7f6cec158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec158cc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6cec069590 tx=0x7f6ce4009840 comp rx=0 tx=0).stop 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 -- 172.21.15.146:0/2452587975 shutdown_connections 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec158cc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 --2- 172.21.15.146:0/2452587975 >> v2:172.21.15.161:3300/0 conn(0x7f6cec0cfb00 0x7f6cec0cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 --2- 172.21.15.146:0/2452587975 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec067030 0x7f6cec067400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 -- 172.21.15.146:0/2452587975 >> 172.21.15.146:0/2452587975 conn(0x7f6cec0d0b50 msgr2=0x7f6cec0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 -- 172.21.15.146:0/2452587975 shutdown_connections 2024-08-15T17:02:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.182+0000 7f6cf1063700 1 -- 172.21.15.146:0/2452587975 wait complete. 2024-08-15T17:02:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 Processor -- start 2024-08-15T17:02:47.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- start start 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec067030 0x7f6cec0b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec0cfb00 0x7f6cec154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b3ce0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec16b5e0 con 0x7f6cec067030 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b3440 con 0x7f6cec0cfb00 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46114/0 (socket says 172.21.15.146:46114) 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 -- 172.21.15.146:0/1037373306 learned_addr learned my addr 172.21.15.146:0/1037373306 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cebfff700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec067030 0x7f6cec0b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf0862700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec0cfb00 0x7f6cec154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 -- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec0cfb00 msgr2=0x7f6cec154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec0cfb00 0x7f6cec154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 -- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec067030 msgr2=0x7f6cec0b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec067030 0x7f6cec0b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 -- 172.21.15.146:0/1037373306 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009ae0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ceb7fe700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400d860 tx=0x7f6cd4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd40125f0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd4012750 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec0b3780 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4012b10 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.203 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.186+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec0b5740 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec0638f0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f6cd401e460 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6ce3fff700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 0x7f6ccc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f6cd407d460 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6cebfff700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 0x7f6ccc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.190+0000 7f6cebfff700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 0x7f6ccc062d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc006fd0 tx=0x7f6cdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.194+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f6cd4050fc0 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.319 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:02:47.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:02:47.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.362+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.data", "var": "pg_num"} v 0) v1 -- 0x7f6cec0b2910 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.362+0000 7f6ce3fff700 1 -- 172.21.15.146:0/1037373306 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.data", "var": "pg_num"}]=0 v37) v1 ==== 124+0+11 (secure 0 0 0) 0x7f6cd4050b50 con 0x7f6cec0cb5c0 2024-08-15T17:02:47.376 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:02:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 msgr2=0x7f6ccc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 0x7f6ccc062d10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc006fd0 tx=0x7f6cdc008040 comp rx=0 tx=0).stop 2024-08-15T17:02:47.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 msgr2=0x7f6cec0b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400d860 tx=0x7f6cd4008040 comp rx=0 tx=0).stop 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 shutdown_connections 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6ccc060860 0x7f6ccc062d10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6cec0cfb00 0x7f6cec154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6cec0cb5c0 0x7f6cec0b5200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 --2- 172.21.15.146:0/1037373306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6cec067030 0x7f6cec0b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 >> 172.21.15.146:0/1037373306 conn(0x7f6cec0d0b50 msgr2=0x7f6cec0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 shutdown_connections 2024-08-15T17:02:47.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.366+0000 7f6cf1063700 1 -- 172.21.15.146:0/1037373306 wait complete. 2024-08-15T17:02:47.394 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:47.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 Processor -- start 2024-08-15T17:02:47.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 -- start start 2024-08-15T17:02:47.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd694155d20 0x7fd6940b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd6940ad7c0 0x7fd6940aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6940af2b0 con 0x7fd69406df10 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940adc10 con 0x7fd6940ad7c0 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd69928b700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940af590 con 0x7fd694155d20 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd6940ad7c0 0x7fd6940aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd6940ad7c0 0x7fd6940aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46128/0 (socket says 172.21.15.146:46128) 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 -- 172.21.15.146:0/828800093 learned_addr learned my addr 172.21.15.146:0/828800093 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd693fff700 1 --2- 172.21.15.146:0/828800093 >> v2:172.21.15.161:3300/0 conn(0x7fd694155d20 0x7fd6940b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 -- 172.21.15.146:0/828800093 >> v2:172.21.15.161:3300/0 conn(0x7fd694155d20 msgr2=0x7fd6940b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 --2- 172.21.15.146:0/828800093 >> v2:172.21.15.161:3300/0 conn(0x7fd694155d20 0x7fd6940b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 -- 172.21.15.146:0/828800093 >> v1:172.21.15.161:6789/0 conn(0x7fd69406df10 legacy=0x7fd69406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 -- 172.21.15.146:0/828800093 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940af870 con 0x7fd6940ad7c0 2024-08-15T17:02:47.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd68bfff700 1 --2- 172.21.15.146:0/828800093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6940ad7c0 0x7fd6940aea80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd6940b1d50 tx=0x7fd67c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90df1a7f8b0f4780 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd6937fe700 1 -- 172.21.15.146:0/828800093 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00dc50 con 0x7fd6940ad7c0 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.482+0000 7fd6937fe700 1 -- 172.21.15.146:0/828800093 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c00ddb0 con 0x7fd6940ad7c0 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd6937fe700 1 -- 172.21.15.146:0/828800093 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c0196d0 con 0x7fd6940ad7c0 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- 172.21.15.146:0/828800093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6940ad7c0 msgr2=0x7fd6940aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- 172.21.15.146:0/828800093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6940ad7c0 0x7fd6940aea80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd6940b1d50 tx=0x7fd67c00b4f0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- 172.21.15.146:0/828800093 shutdown_connections 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- 172.21.15.146:0/828800093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd6940ad7c0 0x7fd6940aea80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- 172.21.15.146:0/828800093 >> v2:172.21.15.161:3300/0 conn(0x7fd694155d20 0x7fd6940b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- 172.21.15.146:0/828800093 >> 172.21.15.146:0/828800093 conn(0x7fd6940678d0 msgr2=0x7fd694067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- 172.21.15.146:0/828800093 shutdown_connections 2024-08-15T17:02:47.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- 172.21.15.146:0/828800093 wait complete. 2024-08-15T17:02:47.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 Processor -- start 2024-08-15T17:02:47.502 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- start start 2024-08-15T17:02:47.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd69406df10 0x7fd694149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 0x7fd694069710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940aa5a0 con 0x7fd694155d20 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940a9990 con 0x7fd69406df10 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd69928b700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940a9c70 con 0x7fd6940ad7c0 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd698a8a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd698a8a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46138/0 (socket says 172.21.15.146:46138) 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd698a8a700 1 -- 172.21.15.146:0/1038671957 learned_addr learned my addr 172.21.15.146:0/1038671957 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd693fff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd69406df10 0x7fd694149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.486+0000 7fd68bfff700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 0x7fd694069710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 -- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 msgr2=0x7fd694069710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 0x7fd694069710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 -- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd69406df10 msgr2=0x7fd694149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd69406df10 0x7fd694149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 -- 172.21.15.146:0/1038671957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67c00b790 con 0x7fd694155d20 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd68bfff700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 0x7fd694069710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd698a8a700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c840 tx=0x7fd68c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c012630 con 0x7fd694155d20 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940a9f50 con 0x7fd694155d20 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c0127b0 con 0x7fd694155d20 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c012b70 con 0x7fd694155d20 2024-08-15T17:02:47.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd694149dd0 con 0x7fd694155d20 2024-08-15T17:02:47.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940aa710 con 0x7fd694155d20 2024-08-15T17:02:47.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.490+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd68c003030 con 0x7fd694155d20 2024-08-15T17:02:47.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.494+0000 7fd690ff9700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 0x7fd674062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.494+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fd68c07cf60 con 0x7fd694155d20 2024-08-15T17:02:47.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.494+0000 7fd693fff700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 0x7fd674062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.494+0000 7fd693fff700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 0x7fd674062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd684006fd0 tx=0x7fd684008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.502+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd68c050a20 con 0x7fd694155d20 2024-08-15T17:02:47.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.698+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6940643a0 con 0x7fd694155d20 2024-08-15T17:02:47.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd690ff9700 1 -- 172.21.15.146:0/1038671957 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5692 (secure 0 0 0) 0x7fd68c0505b0 con 0x7fd694155d20 2024-08-15T17:02:47.716 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:47.716 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 msgr2=0x7fd674062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 0x7fd674062d10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd684006fd0 tx=0x7fd684008040 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 msgr2=0x7fd694069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c840 tx=0x7fd68c001670 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 shutdown_connections 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd674060860 0x7fd674062d10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd694155d20 0x7fd694069c50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6940ad7c0 0x7fd694069710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 --2- 172.21.15.146:0/1038671957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd69406df10 0x7fd694149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 >> 172.21.15.146:0/1038671957 conn(0x7fd6940678d0 msgr2=0x7fd694063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 shutdown_connections 2024-08-15T17:02:47.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.702+0000 7fd69928b700 1 -- 172.21.15.146:0/1038671957 wait complete. 2024-08-15T17:02:47.720 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-08-15T17:02:47.731 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:47.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 Processor -- start 2024-08-15T17:02:47.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 -- start start 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c0686a0 0x7fee2c068af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee2c069c50 0x7fee2c06a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2c06af40 con 0x7fee2c155570 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2c06b7e0 con 0x7fee2c069c50 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee332a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c069030 con 0x7fee2c0686a0 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee2c069c50 0x7fee2c06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fee2c069c50 0x7fee2c06a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46152/0 (socket says 172.21.15.146:46152) 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 -- 172.21.15.146:0/2952288298 learned_addr learned my addr 172.21.15.146:0/2952288298 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee30844700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c0686a0 0x7fee2c068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 -- 172.21.15.146:0/2952288298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c0686a0 msgr2=0x7fee2c068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c0686a0 0x7fee2c068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 -- 172.21.15.146:0/2952288298 >> v1:172.21.15.161:6789/0 conn(0x7fee2c155570 legacy=0x7fee2c06bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 -- 172.21.15.146:0/2952288298 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2c06a9c0 con 0x7fee2c069c50 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee31846700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c069c50 0x7fee2c06a090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee240099b0 tx=0x7fee24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1744fa3605fa4f21 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee23fff700 1 -- 172.21.15.146:0/2952288298 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400b860 con 0x7fee2c069c50 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee23fff700 1 -- 172.21.15.146:0/2952288298 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2400b9c0 con 0x7fee2c069c50 2024-08-15T17:02:47.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.822+0000 7fee23fff700 1 -- 172.21.15.146:0/2952288298 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2400bd80 con 0x7fee2c069c50 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- 172.21.15.146:0/2952288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c069c50 msgr2=0x7fee2c06a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c069c50 0x7fee2c06a090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fee240099b0 tx=0x7fee24009250 comp rx=0 tx=0).stop 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- 172.21.15.146:0/2952288298 shutdown_connections 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c069c50 0x7fee2c06a090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 --2- 172.21.15.146:0/2952288298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c0686a0 0x7fee2c068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- 172.21.15.146:0/2952288298 >> 172.21.15.146:0/2952288298 conn(0x7fee2c06db30 msgr2=0x7fee2c06df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:47.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- 172.21.15.146:0/2952288298 shutdown_connections 2024-08-15T17:02:47.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- 172.21.15.146:0/2952288298 wait complete. 2024-08-15T17:02:47.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 Processor -- start 2024-08-15T17:02:47.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.826+0000 7fee332a9700 1 -- start start 2024-08-15T17:02:47.846 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 0x7fee2c0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c155570 0x7fee2c169890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c148f50 con 0x7fee2c0686a0 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c14b900 con 0x7fee2c069c50 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2c14bbe0 con 0x7fee2c155570 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46164/0 (socket says 172.21.15.146:46164) 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 -- 172.21.15.146:0/3952152313 learned_addr learned my addr 172.21.15.146:0/3952152313 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee30844700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 0x7fee2c0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31846700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c155570 0x7fee2c169890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.847 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 -- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c155570 msgr2=0x7fee2c169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c155570 0x7fee2c169890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 -- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 msgr2=0x7fee2c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 0x7fee2c0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 -- 172.21.15.146:0/3952152313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee240094f0 con 0x7fee2c0686a0 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee30844700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 0x7fee2c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee31045700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9d0 tx=0x7fee1c00c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0018a0 con 0x7fee2c0686a0 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c001a00 con 0x7fee2c0686a0 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2c14bec0 con 0x7fee2c0686a0 2024-08-15T17:02:47.848 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c011470 con 0x7fee2c0686a0 2024-08-15T17:02:47.849 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2c0afdb0 con 0x7fee2c0686a0 2024-08-15T17:02:47.850 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.830+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2c0645c0 con 0x7fee2c0686a0 2024-08-15T17:02:47.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.834+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fee1c017030 con 0x7fee2c0686a0 2024-08-15T17:02:47.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.834+0000 7fee217fa700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 0x7fee18062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:47.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.834+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fee1c015070 con 0x7fee2c0686a0 2024-08-15T17:02:47.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.834+0000 7fee30844700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 0x7fee18062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:47.851 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.834+0000 7fee30844700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 0x7fee18062d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee2c152960 tx=0x7fee2800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:47.856 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:47.842+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fee1c04f3d0 con 0x7fee2c0686a0 2024-08-15T17:02:48.016 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.002+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee2c06ed80 con 0x7fee2c0686a0 2024-08-15T17:02:48.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.002+0000 7fee217fa700 1 -- 172.21.15.146:0/3952152313 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18647 (secure 0 0 0) 0x7fee1c0529f0 con 0x7fee2c0686a0 2024-08-15T17:02:48.019 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:48.019 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:45.834731+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:48.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 msgr2=0x7fee18062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 0x7fee18062d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee2c152960 tx=0x7fee2800b040 comp rx=0 tx=0).stop 2024-08-15T17:02:48.021 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 msgr2=0x7fee2c148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00e9d0 tx=0x7fee1c00c300 comp rx=0 tx=0).stop 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 shutdown_connections 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fee18060890 0x7fee18062d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fee2c155570 0x7fee2c169890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fee2c069c50 0x7fee2c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 --2- 172.21.15.146:0/3952152313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fee2c0686a0 0x7fee2c148a10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 >> 172.21.15.146:0/3952152313 conn(0x7fee2c06db30 msgr2=0x7fee2c158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 shutdown_connections 2024-08-15T17:02:48.022 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.006+0000 7fee332a9700 1 -- 172.21.15.146:0/3952152313 wait complete. 2024-08-15T17:02:48.033 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:02:48.033 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:48.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 Processor -- start 2024-08-15T17:02:48.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- start start 2024-08-15T17:02:48.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc2540c86a0 0x7fc2540cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2540c2e00 0x7fc254068600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc254068b40 0x7fc2540c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540ca410 con 0x7fc2540c86a0 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2540cdd30 con 0x7fc254068b40 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2540c3e80 con 0x7fc2540c2e00 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc2540c86a0 0x7fc2540cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc2540c86a0 0x7fc2540cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59418/0 (socket says 172.21.15.146:59418) 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 -- 172.21.15.146:0/444841574 learned_addr learned my addr 172.21.15.146:0/444841574 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25898f700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2540c2e00 0x7fc254068600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259991700 1 --2- 172.21.15.146:0/444841574 >> v2:172.21.15.161:3300/0 conn(0x7fc254068b40 0x7fc2540c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 -- 172.21.15.146:0/444841574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2540c2e00 msgr2=0x7fc254068600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:48.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2540c2e00 0x7fc254068600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 -- 172.21.15.146:0/444841574 >> v2:172.21.15.161:3300/0 conn(0x7fc254068b40 msgr2=0x7fc2540c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 --2- 172.21.15.146:0/444841574 >> v2:172.21.15.161:3300/0 conn(0x7fc254068b40 0x7fc2540c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 -- 172.21.15.146:0/444841574 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540c3270 con 0x7fc2540c86a0 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc259190700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c86a0 0x7fc2540cbdc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc25406f130 tx=0x7fc248009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9875ebe26a04e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc247fff700 1 -- 172.21.15.146:0/444841574 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800b640 con 0x7fc2540c86a0 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc247fff700 1 -- 172.21.15.146:0/444841574 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24800b7a0 con 0x7fc2540c86a0 2024-08-15T17:02:48.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc247fff700 1 -- 172.21.15.146:0/444841574 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800bb60 con 0x7fc2540c86a0 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- 172.21.15.146:0/444841574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c86a0 msgr2=0x7fc2540cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c86a0 0x7fc2540cbdc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc25406f130 tx=0x7fc248009250 comp rx=0 tx=0).stop 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 -- 172.21.15.146:0/444841574 shutdown_connections 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/444841574 >> v2:172.21.15.161:3300/0 conn(0x7fc254068b40 0x7fc2540c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2540c2e00 0x7fc254068600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.126+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/444841574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c86a0 0x7fc2540cbdc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- 172.21.15.146:0/444841574 >> 172.21.15.146:0/444841574 conn(0x7fc2540caad0 msgr2=0x7fc2540caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- 172.21.15.146:0/444841574 shutdown_connections 2024-08-15T17:02:48.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- 172.21.15.146:0/444841574 wait complete. 2024-08-15T17:02:48.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 Processor -- start 2024-08-15T17:02:48.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- start start 2024-08-15T17:02:48.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc254068b40 0x7fc2540b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 0x7fc2540b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406c4b0 con 0x7fc2540c2e00 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406b8a0 con 0x7fc2540b0e60 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25b3f4700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25406bb80 con 0x7fc254068b40 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59430/0 (socket says 172.21.15.146:59430) 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 -- 172.21.15.146:0/102151116 learned_addr learned my addr 172.21.15.146:0/102151116 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc259190700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc254068b40 0x7fc2540b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc259991700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 0x7fc2540b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.149 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 -- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc254068b40 msgr2=0x7fc2540b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc254068b40 0x7fc2540b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 -- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 msgr2=0x7fc2540b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 0x7fc2540b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc25898f700 1 -- 172.21.15.146:0/102151116 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480094f0 con 0x7fc2540c2e00 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.130+0000 7fc259991700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 0x7fc2540b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc25898f700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009ef0 tx=0x7fc24c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c00cc20 con 0x7fc2540c2e00 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c00f040 con 0x7fc2540c2e00 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25406be60 con 0x7fc2540c2e00 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c014480 con 0x7fc2540c2e00 2024-08-15T17:02:48.150 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25406b560 con 0x7fc2540c2e00 2024-08-15T17:02:48.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2540713c0 con 0x7fc2540c2e00 2024-08-15T17:02:48.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc24c0147f0 con 0x7fc2540c2e00 2024-08-15T17:02:48.151 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.134+0000 7fc2457fa700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 0x7fc238062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.138+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fc24c07d920 con 0x7fc2540c2e00 2024-08-15T17:02:48.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.138+0000 7fc259190700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 0x7fc238062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.152 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.138+0000 7fc259190700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 0x7fc238062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc25406f130 tx=0x7fc248017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.158 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.142+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc2540713c0 con 0x7fc2540c2e00 2024-08-15T17:02:48.348 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.334+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc25414b020 con 0x7fc2540c2e00 2024-08-15T17:02:48.350 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.334+0000 7fc2457fa700 1 -- 172.21.15.146:0/102151116 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5692 (secure 0 0 0) 0x7fc25414b020 con 0x7fc2540c2e00 2024-08-15T17:02:48.350 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:48.350 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:48.352 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 msgr2=0x7fc238062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 0x7fc238062d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc25406f130 tx=0x7fc248017000 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 msgr2=0x7fc25406b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009ef0 tx=0x7fc24c00b040 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 shutdown_connections 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc238060860 0x7fc238062d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2540b0e60 0x7fc2540b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc2540c2e00 0x7fc25406b020 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 --2- 172.21.15.146:0/102151116 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc254068b40 0x7fc2540b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 >> 172.21.15.146:0/102151116 conn(0x7fc2540caad0 msgr2=0x7fc2541611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 shutdown_connections 2024-08-15T17:02:48.353 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.338+0000 7fc25b3f4700 1 -- 172.21.15.146:0/102151116 wait complete. 2024-08-15T17:02:48.354 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-08-15T17:02:48.364 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:48.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.454+0000 7f8c45882700 1 Processor -- start 2024-08-15T17:02:48.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.454+0000 7f8c45882700 1 -- start start 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45882700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c400b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45882700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45882700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400ae700 con 0x7f8c400ad660 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45882700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c400adaf0 con 0x7f8c40068db0 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45882700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c400addd0 con 0x7f8c401588c0 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c400b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45081700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45081700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:50082/0 (socket says 172.21.15.146:50082) 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45081700 1 -- 172.21.15.146:0/1572268980 learned_addr learned my addr 172.21.15.146:0/1572268980 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.473 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c37fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8c400ad660 0x7f8c400686a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:52886/0 (socket says 172.21.15.146:52886) 2024-08-15T17:02:48.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 -- 172.21.15.146:0/1572268980 >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 msgr2=0x7f8c40155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:48.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 --2- 172.21.15.146:0/1572268980 >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 -- 172.21.15.146:0/1572268980 >> v1:172.21.15.161:6789/0 conn(0x7f8c400ad660 legacy=0x7f8c400686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 -- 172.21.15.146:0/1572268980 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c400ae0b0 con 0x7f8c401588c0 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c45081700 1 --2- 172.21.15.146:0/1572268980 >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c44880700 1 --2- 172.21.15.146:0/1572268980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c400b1ec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009910 tx=0x7f8c3c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ebce8898d9c4443 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c377fe700 1 -- 172.21.15.146:0/1572268980 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00f6b0 con 0x7f8c401588c0 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c377fe700 1 -- 172.21.15.146:0/1572268980 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00f810 con 0x7f8c401588c0 2024-08-15T17:02:48.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.458+0000 7f8c377fe700 1 -- 172.21.15.146:0/1572268980 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00fbd0 con 0x7f8c401588c0 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- 172.21.15.146:0/1572268980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 msgr2=0x7f8c400b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- 172.21.15.146:0/1572268980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c400b1ec0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009910 tx=0x7f8c3c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- 172.21.15.146:0/1572268980 shutdown_connections 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- 172.21.15.146:0/1572268980 >> v2:172.21.15.161:3300/0 conn(0x7f8c40068db0 0x7f8c40155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- 172.21.15.146:0/1572268980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c400b1ec0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- 172.21.15.146:0/1572268980 >> 172.21.15.146:0/1572268980 conn(0x7f8c4006d870 msgr2=0x7f8c4006dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- 172.21.15.146:0/1572268980 shutdown_connections 2024-08-15T17:02:48.476 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- 172.21.15.146:0/1572268980 wait complete. 2024-08-15T17:02:48.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 Processor -- start 2024-08-15T17:02:48.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 -- start start 2024-08-15T17:02:48.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c40068db0 0x7f8c4014c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.480 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.462+0000 7f8c45882700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c40067dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c401488f0 con 0x7f8c400ad660 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4014d960 con 0x7f8c40068db0 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4014e670 con 0x7f8c401588c0 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59436/0 (socket says 172.21.15.146:59436) 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45081700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c40067dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 -- 172.21.15.146:0/1356526214 learned_addr learned my addr 172.21.15.146:0/1356526214 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c44880700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c40068db0 0x7f8c4014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 -- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 msgr2=0x7f8c40067dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c40067dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 -- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c40068db0 msgr2=0x7f8c4014c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c40068db0 0x7f8c4014c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.481 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 -- 172.21.15.146:0/1356526214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3800abf0 con 0x7f8c400ad660 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c37fff700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c7f0 tx=0x7f8c30007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30005a70 con 0x7f8c400ad660 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30005bf0 con 0x7f8c400ad660 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c009420 con 0x7f8c400ad660 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300070d0 con 0x7f8c400ad660 2024-08-15T17:02:48.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4014e020 con 0x7f8c400ad660 2024-08-15T17:02:48.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c400638f0 con 0x7f8c400ad660 2024-08-15T17:02:48.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.466+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f8c30007480 con 0x7f8c400ad660 2024-08-15T17:02:48.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.470+0000 7f8c34ff9700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 0x7f8c2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.470+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f8c3007c340 con 0x7f8c400ad660 2024-08-15T17:02:48.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.470+0000 7f8c44880700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 0x7f8c2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.470+0000 7f8c44880700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 0x7f8c2c062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0093f0 tx=0x7f8c3c019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.491 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.474+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f8c3004c410 con 0x7f8c400ad660 2024-08-15T17:02:48.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c4014e7b0 con 0x7f8c400ad660 2024-08-15T17:02:48.688 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c34ff9700 1 -- 172.21.15.146:0/1356526214 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5692 (secure 0 0 0) 0x7f8c3004fa30 con 0x7f8c400ad660 2024-08-15T17:02:48.689 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:48.689 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:28.770829+0000","modified":"2024-08-15T17:00:32.438138+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"d","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2559773181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2559773181},{"type":"v1","addr":"172.21.15.161:6837","nonce":2559773181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:48.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 msgr2=0x7f8c2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 0x7f8c2c062d10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0093f0 tx=0x7f8c3c019000 comp rx=0 tx=0).stop 2024-08-15T17:02:48.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 msgr2=0x7f8c40148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.691 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000c7f0 tx=0x7f8c30007f70 comp rx=0 tx=0).stop 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 shutdown_connections 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f8c2c060860 0x7f8c2c062d10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c401588c0 0x7f8c40067dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c400ad660 0x7f8c40148a70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 --2- 172.21.15.146:0/1356526214 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c40068db0 0x7f8c4014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.674+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 >> 172.21.15.146:0/1356526214 conn(0x7f8c4006d870 msgr2=0x7f8c40158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.678+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 shutdown_connections 2024-08-15T17:02:48.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.678+0000 7f8c45882700 1 -- 172.21.15.146:0/1356526214 wait complete. 2024-08-15T17:02:48.693 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 18 2024-08-15T17:02:48.703 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:48.806 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.790+0000 7fa179b49700 1 Processor -- start 2024-08-15T17:02:48.807 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.790+0000 7fa179b49700 1 -- start start 2024-08-15T17:02:48.808 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1740ab540 0x7fa17406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa174155720 0x7fa1740b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1740aba70 con 0x7fa17406e9d0 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1740abd50 con 0x7fa174155720 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740ac030 con 0x7fa1740ab540 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa174155720 0x7fa1740b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa174155720 0x7fa1740b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59446/0 (socket says 172.21.15.146:59446) 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 -- 172.21.15.146:0/738549881 learned_addr learned my addr 172.21.15.146:0/738549881 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa178b47700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fa17406e9d0 0x7fa174155010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49644/0 (socket says 172.21.15.146:49644) 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa173fff700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1740ab540 0x7fa17406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 -- 172.21.15.146:0/738549881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1740ab540 msgr2=0x7fa17406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1740ab540 0x7fa17406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.809 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 -- 172.21.15.146:0/738549881 >> v1:172.21.15.146:6789/0 conn(0x7fa17406e9d0 legacy=0x7fa174155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 -- 172.21.15.146:0/738549881 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740ac310 con 0x7fa174155720 2024-08-15T17:02:48.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179348700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa174155720 0x7fa1740b1320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0099b0 tx=0x7fa16c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380ea7e1b425ea21 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa1737fe700 1 -- 172.21.15.146:0/738549881 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00b860 con 0x7fa174155720 2024-08-15T17:02:48.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa1737fe700 1 -- 172.21.15.146:0/738549881 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c00b9c0 con 0x7fa174155720 2024-08-15T17:02:48.810 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa1737fe700 1 -- 172.21.15.146:0/738549881 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00bd60 con 0x7fa174155720 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- 172.21.15.146:0/738549881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa174155720 msgr2=0x7fa1740b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa174155720 0x7fa1740b1320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0099b0 tx=0x7fa16c009250 comp rx=0 tx=0).stop 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- 172.21.15.146:0/738549881 shutdown_connections 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa174155720 0x7fa1740b1320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 --2- 172.21.15.146:0/738549881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1740ab540 0x7fa17406eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- 172.21.15.146:0/738549881 >> 172.21.15.146:0/738549881 conn(0x7fa1740678d0 msgr2=0x7fa174067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- 172.21.15.146:0/738549881 shutdown_connections 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- 172.21.15.146:0/738549881 wait complete. 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 Processor -- start 2024-08-15T17:02:48.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.794+0000 7fa179b49700 1 -- start start 2024-08-15T17:02:48.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1740ab540 0x7fa174152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.813 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa174155720 0x7fa1741544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1741531a0 con 0x7fa17406e9d0 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa174153440 con 0x7fa1740ab540 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa174153720 con 0x7fa174155720 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59462/0 (socket says 172.21.15.146:59462) 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 -- 172.21.15.146:0/2601964082 learned_addr learned my addr 172.21.15.146:0/2601964082 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179348700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa174155720 0x7fa1741544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa173fff700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1740ab540 0x7fa174152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 -- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa174155720 msgr2=0x7fa1741544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa174155720 0x7fa1741544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 -- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1740ab540 msgr2=0x7fa174152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1740ab540 0x7fa174152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 -- 172.21.15.146:0/2601964082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0094f0 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa178b47700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c7c0 tx=0x7fa164007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164005a70 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164005bd0 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa174153a00 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640070d0 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa174153fc0 con 0x7fa17406e9d0 2024-08-15T17:02:48.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.798+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1741541b0 con 0x7fa17406e9d0 2024-08-15T17:02:48.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.802+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa164007480 con 0x7fa17406e9d0 2024-08-15T17:02:48.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.802+0000 7fa170ff9700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 0x7fa154062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:48.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.802+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fa16407c400 con 0x7fa17406e9d0 2024-08-15T17:02:48.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.802+0000 7fa173fff700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 0x7fa154062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:48.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.802+0000 7fa173fff700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 0x7fa154062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009fd0 tx=0x7fa15c009880 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:48.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.810+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa16404c430 con 0x7fa17406e9d0 2024-08-15T17:02:48.989 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.974+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1740643a0 con 0x7fa17406e9d0 2024-08-15T17:02:48.991 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.974+0000 7fa170ff9700 1 -- 172.21.15.146:0/2601964082 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18647 (secure 0 0 0) 0x7fa16404fa50 con 0x7fa17406e9d0 2024-08-15T17:02:48.991 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:48.991 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":37,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:00:45.834731+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-15T17:00:27.040042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-15T17:00:27.765439+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 msgr2=0x7fa154062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 0x7fa154062d10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009fd0 tx=0x7fa15c009880 comp rx=0 tx=0).stop 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 msgr2=0x7fa17413b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c7c0 tx=0x7fa164007f70 comp rx=0 tx=0).stop 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 shutdown_connections 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa154060860 0x7fa154062d10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa174155720 0x7fa1741544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1740ab540 0x7fa174152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 --2- 172.21.15.146:0/2601964082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa17406e9d0 0x7fa17413b990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:48.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 >> 172.21.15.146:0/2601964082 conn(0x7fa1740678d0 msgr2=0x7fa17406d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:48.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 shutdown_connections 2024-08-15T17:02:48.995 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:48.978+0000 7fa179b49700 1 -- 172.21.15.146:0/2601964082 wait complete. 2024-08-15T17:02:49.006 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-15T17:02:49.112 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 Processor -- start 2024-08-15T17:02:49.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 -- start start 2024-08-15T17:02:49.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf340b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf340686a0 0x7fdf34068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf340ad970 con 0x7fdf34155530 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf340abee0 con 0x7fdf340686a0 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf395a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34069060 con 0x7fdf340aba90 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf337fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fdf340686a0 0x7fdf34068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf32ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fdf34155530 0x7fdf34155900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49652/0 (socket says 172.21.15.146:49652) 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf32ffd700 1 -- 172.21.15.146:0/3288197748 learned_addr learned my addr 172.21.15.146:0/3288197748 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:49.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf327fc700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf340b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf337fe700 1 -- 172.21.15.146:0/3288197748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 msgr2=0x7fdf340b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf337fe700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf340b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf337fe700 1 -- 172.21.15.146:0/3288197748 >> v1:172.21.15.146:6789/0 conn(0x7fdf34155530 legacy=0x7fdf34155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf337fe700 1 -- 172.21.15.146:0/3288197748 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34156010 con 0x7fdf340686a0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.098+0000 7fdf31ffb700 1 -- 172.21.15.146:0/3288197748 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522634127 0 0) 0x7fdf340ad970 con 0x7fdf34155530 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf337fe700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34068b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009e90 tx=0x7fdf2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731ac3e9f7ff1df4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf31ffb700 1 -- 172.21.15.146:0/3288197748 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00e070 con 0x7fdf340686a0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf31ffb700 1 -- 172.21.15.146:0/3288197748 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c00bbf0 con 0x7fdf340686a0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf31ffb700 1 -- 172.21.15.146:0/3288197748 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c012430 con 0x7fdf340686a0 2024-08-15T17:02:49.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- 172.21.15.146:0/3288197748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 msgr2=0x7fdf34068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34068b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009e90 tx=0x7fdf2c009780 comp rx=0 tx=0).stop 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- 172.21.15.146:0/3288197748 shutdown_connections 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34068b20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 --2- 172.21.15.146:0/3288197748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf340b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- 172.21.15.146:0/3288197748 >> 172.21.15.146:0/3288197748 conn(0x7fdf3406d940 msgr2=0x7fdf3406dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- 172.21.15.146:0/3288197748 shutdown_connections 2024-08-15T17:02:49.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- 172.21.15.146:0/3288197748 wait complete. 2024-08-15T17:02:49.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 Processor -- start 2024-08-15T17:02:49.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.102+0000 7fdf395a3700 1 -- start start 2024-08-15T17:02:49.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf341584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34155530 0x7fdf3416ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34157910 con 0x7fdf340686a0 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34159760 con 0x7fdf34155530 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34158b50 con 0x7fdf340aba90 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59486/0 (socket says 172.21.15.146:59486) 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf327fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf341584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 -- 172.21.15.146:0/611798221 learned_addr learned my addr 172.21.15.146:0/611798221 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:49.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf337fe700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34155530 0x7fdf3416ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 -- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 msgr2=0x7fdf341584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf341584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 -- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34155530 msgr2=0x7fdf3416ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34155530 0x7fdf3416ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 -- 172.21.15.146:0/611798221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c009a20 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf32ffd700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c5b0 tx=0x7fdf24001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24005b30 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24005c90 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf34158e30 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24007300 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf341593f0 con 0x7fdf340686a0 2024-08-15T17:02:49.123 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.106+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf340645c0 con 0x7fdf340686a0 2024-08-15T17:02:49.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.110+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fdf2401b3b0 con 0x7fdf340686a0 2024-08-15T17:02:49.125 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.110+0000 7fdf2affd700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 0x7fdf14062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.110+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fdf2407c370 con 0x7fdf340686a0 2024-08-15T17:02:49.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.110+0000 7fdf327fc700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 0x7fdf14062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.126 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.110+0000 7fdf327fc700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 0x7fdf14062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf34157e30 tx=0x7fdf1c00a330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.118+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fdf2404c3a0 con 0x7fdf340686a0 2024-08-15T17:02:49.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.318+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdf340ad360 con 0x7fdf340686a0 2024-08-15T17:02:49.335 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-08-15T17:02:49.322+0000 7fbf03872700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:02:49.343 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.326+0000 7fdf2affd700 1 -- 172.21.15.146:0/611798221 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdf2404f9c0 con 0x7fdf340686a0 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 msgr2=0x7fdf14062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 0x7fdf14062d10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf34157e30 tx=0x7fdf1c00a330 comp rx=0 tx=0).stop 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 msgr2=0x7fdf34157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c5b0 tx=0x7fdf24001070 comp rx=0 tx=0).stop 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 shutdown_connections 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fdf14060860 0x7fdf14062d10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf34155530 0x7fdf3416ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.346 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf340aba90 0x7fdf341584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 --2- 172.21.15.146:0/611798221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdf340686a0 0x7fdf34157f70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 >> 172.21.15.146:0/611798221 conn(0x7fdf3406d940 msgr2=0x7fdf34069df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:49.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 shutdown_connections 2024-08-15T17:02:49.347 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.330+0000 7fdf395a3700 1 -- 172.21.15.146:0/611798221 wait complete. 2024-08-15T17:02:49.347 INFO:teuthology.orchestra.run.smithi146.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:02:49.351 INFO:tasks.ceph.mds.d.smithi161.stderr: -7216> 2024-08-15T17:00:34.149+0000 7f48258d2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-15T17:02:49.359 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-15T17:02:49.380 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-08-15T17:02:49.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 Processor -- start 2024-08-15T17:02:49.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 -- start start 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2cd01514d0 0x7f2cd006ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd0157960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd00b1750 con 0x7f2cd00b0d90 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd006e010 con 0x7f2cd01514d0 2024-08-15T17:02:49.474 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd006efa0 con 0x7f2cd01574f0 2024-08-15T17:02:49.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccf7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd0157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccf7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59500/0 (socket says 172.21.15.146:59500) 2024-08-15T17:02:49.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cceffd700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.161:3300/0 conn(0x7f2cd01514d0 0x7f2cd006ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.475 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccf7fe700 1 -- 172.21.15.146:0/1907578661 learned_addr learned my addr 172.21.15.146:0/1907578661 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 -- 172.21.15.146:0/1907578661 >> v2:172.21.15.161:3300/0 conn(0x7f2cd01514d0 msgr2=0x7f2cd006ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.161:3300/0 conn(0x7f2cd01514d0 0x7f2cd006ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 -- 172.21.15.146:0/1907578661 >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 msgr2=0x7f2cd00b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 -- 172.21.15.146:0/1907578661 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0157fb0 con 0x7f2cd01574f0 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2ccf7fe700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.458+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/1907578661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd0157960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cd005f080 tx=0x7f2cc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649995652bd98da1 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccdffb700 1 -- 172.21.15.146:0/1907578661 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011630 con 0x7f2cd01574f0 2024-08-15T17:02:49.477 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccdffb700 1 -- 172.21.15.146:0/1907578661 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40117b0 con 0x7f2cd01574f0 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccdffb700 1 -- 172.21.15.146:0/1907578661 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4011b70 con 0x7f2cd01574f0 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- 172.21.15.146:0/1907578661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 msgr2=0x7f2cd0157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 --2- 172.21.15.146:0/1907578661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd0157960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cd005f080 tx=0x7f2cc40091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- 172.21.15.146:0/1907578661 shutdown_connections 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.146:3300/0 conn(0x7f2cd00b0d90 0x7f2cd00b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 --2- 172.21.15.146:0/1907578661 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd0157960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 --2- 172.21.15.146:0/1907578661 >> v2:172.21.15.161:3300/0 conn(0x7f2cd01514d0 0x7f2cd006ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- 172.21.15.146:0/1907578661 >> 172.21.15.146:0/1907578661 conn(0x7f2cd00678a0 msgr2=0x7f2cd0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:49.478 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- 172.21.15.146:0/1907578661 shutdown_connections 2024-08-15T17:02:49.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- 172.21.15.146:0/1907578661 wait complete. 2024-08-15T17:02:49.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 Processor -- start 2024-08-15T17:02:49.479 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.462+0000 7f2ccffff700 1 -- start start 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 0x7f2cd0151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd0140ce0 con 0x7f2cd01514d0 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd014d510 con 0x7f2cd00b0d90 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd014c630 con 0x7f2cd01574f0 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59502/0 (socket says 172.21.15.146:59502) 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 -- 172.21.15.146:0/480269980 learned_addr learned my addr 172.21.15.146:0/480269980 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccf7fe700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cceffd700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 0x7f2cd0151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 -- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 msgr2=0x7f2cd014cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd014cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 -- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 msgr2=0x7f2cd0151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 0x7f2cd0151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 -- 172.21.15.146:0/480269980 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8009ae0 con 0x7f2cd01514d0 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cceffd700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 0x7f2cd0151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cce7fc700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ea0 tx=0x7f2cc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4000d10 con 0x7f2cd01514d0 2024-08-15T17:02:49.483 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4011b30 con 0x7f2cd01514d0 2024-08-15T17:02:49.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4009420 con 0x7f2cd01514d0 2024-08-15T17:02:49.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc401a660 con 0x7f2cd01514d0 2024-08-15T17:02:49.484 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd014cbf0 con 0x7f2cd01514d0 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.466+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd0064330 con 0x7f2cd01514d0 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.470+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2cc401aa10 con 0x7f2cd01514d0 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.470+0000 7f2cb6ffd700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 0x7f2cbc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.470+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f2cc407ebe0 con 0x7f2cd01514d0 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.470+0000 7f2cceffd700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 0x7f2cbc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:49.485 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.470+0000 7f2cceffd700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 0x7f2cbc062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006f00 tx=0x7f2cc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:49.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.474+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f2cc404ec50 con 0x7f2cd01514d0 2024-08-15T17:02:49.692 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:49.678+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2cd00cae20 con 0x7f2cd01514d0 2024-08-15T17:02:50.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.338+0000 7f2cb6ffd700 1 -- 172.21.15.146:0/480269980 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2cc4052270 con 0x7f2cd01514d0 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 msgr2=0x7f2cbc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 0x7f2cbc062d10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0006f00 tx=0x7f2cc0008040 comp rx=0 tx=0).stop 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 msgr2=0x7f2cd01407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4009ea0 tx=0x7f2cc4017000 comp rx=0 tx=0).stop 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 shutdown_connections 2024-08-15T17:02:50.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2cbc060860 0x7f2cbc062d10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd01574f0 0x7f2cd014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2cd01514d0 0x7f2cd01407a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 --2- 172.21.15.146:0/480269980 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cd00b0d90 0x7f2cd0151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 >> 172.21.15.146:0/480269980 conn(0x7f2cd00678a0 msgr2=0x7f2cd0063060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.342+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 shutdown_connections 2024-08-15T17:02:50.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.346+0000 7f2ccffff700 1 -- 172.21.15.146:0/480269980 wait complete. 2024-08-15T17:02:50.376 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-15T17:02:50.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.478+0000 7f66b25a3700 1 Processor -- start 2024-08-15T17:02:50.495 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.478+0000 7f66b25a3700 1 -- start start 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b25a3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f66ac0c6160 0x7f66ac0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b25a3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ac0cc480 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b25a3700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ac159720 con 0x7f66ac0cd2b0 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b25a3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac0cb870 con 0x7f66ac0c6160 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b0b40700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f66ac0cb3f0 0x7f66ac0ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49656/0 (socket says 172.21.15.146:49656) 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b0b40700 1 -- 172.21.15.146:0/3596560007 learned_addr learned my addr 172.21.15.146:0/3596560007 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/3596560007 >> v2:172.21.15.161:3300/0 conn(0x7f66ac0c6160 0x7f66ac0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826573564 0 0) 0x7f66ac0cc480 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6694003570 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652774827 0 0) 0x7f6694003570 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 >> v2:172.21.15.161:3300/0 conn(0x7f66ac0c6160 msgr2=0x7f66ac0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 --2- 172.21.15.146:0/3596560007 >> v2:172.21.15.161:3300/0 conn(0x7f66ac0c6160 0x7f66ac0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 >> v1:172.21.15.161:6789/0 conn(0x7f66ac0cd2b0 legacy=0x7f66ac0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0cbb50 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f66a40017f0 con 0x7f66ac0cb3f0 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 >> v1:172.21.15.146:6789/0 conn(0x7f66ac0cb3f0 legacy=0x7f66ac0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6694003a60 0x7f6694003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f6694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66940051c0 0x7f669400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0cb870 con 0x7f6694004810 2024-08-15T17:02:50.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669400c0d0 con 0x7f66940051c0 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669400c290 con 0x7f6694003a60 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f6694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66abfff700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6694003a60 0x7f6694003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b0b40700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66940051c0 0x7f669400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 -- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6694003a60 msgr2=0x7f6694003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6694003a60 0x7f6694003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 -- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66940051c0 msgr2=0x7f669400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66940051c0 0x7f669400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 -- 172.21.15.146:0/3596560007 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694003570 con 0x7f6694004810 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66b0b40700 1 -- 172.21.15.146:0/3596560007 reap_dead start 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f6694004c80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f669c009c00 tx=0x7f669c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0073318913fe28 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:50.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00e710 con 0x7f6694004810 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c00e870 con 0x7f6694004810 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.482+0000 7f66aaffd700 1 -- 172.21.15.146:0/3596560007 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c00ec30 con 0x7f6694004810 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 msgr2=0x7f6694004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f6694004c80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f669c009c00 tx=0x7f669c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- 172.21.15.146:0/3596560007 shutdown_connections 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 --2- 172.21.15.146:0/3596560007 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f6694004c80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- 172.21.15.146:0/3596560007 >> 172.21.15.146:0/3596560007 conn(0x7f66ac099550 msgr2=0x7f66ac067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- 172.21.15.146:0/3596560007 shutdown_connections 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- 172.21.15.146:0/3596560007 wait complete. 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 Processor -- start 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- start start 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f66ac0b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 0x7f66ac154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 0x7f66ac1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0b3730 con 0x7f6694004810 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0b42b0 con 0x7f66ac157370 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b25a3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac157d00 con 0x7f66ac0c7d20 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f66ac0b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66ab7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 0x7f66ac154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b0b40700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 0x7f66ac1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66ab7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 0x7f66ac154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:54200/0 (socket says 172.21.15.146:54200) 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66ab7fe700 1 -- 172.21.15.146:0/1975424199 learned_addr learned my addr 172.21.15.146:0/1975424199 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 -- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 msgr2=0x7f66ac154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 0x7f66ac154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 -- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 msgr2=0x7f66ac1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:50.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 0x7f66ac1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66abfff700 1 -- 172.21.15.146:0/1975424199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0c8190 con 0x7f6694004810 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.486+0000 7f66b0b40700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 0x7f66ac1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f66abfff700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f66ac0b5be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0d0280 tx=0x7f66a0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0013420 con 0x7f6694004810 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0013580 con 0x7f6694004810 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac0b4740 con 0x7f6694004810 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0013900 con 0x7f6694004810 2024-08-15T17:02:50.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0b3980 con 0x7f6694004810 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ac0638f0 con 0x7f6694004810 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f66a0013cf0 con 0x7f6694004810 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 0x7f6698062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f66a007cdc0 con 0x7f6694004810 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.490+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 0x7f6698062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:50.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.494+0000 7f66ab7fe700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 0x7f6698062d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c01a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:50.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.498+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f66a004ce30 con 0x7f6694004810 2024-08-15T17:02:50.680 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:50.666+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66ac068020 con 0x7f6694004810 2024-08-15T17:02:51.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.342+0000 7f6693fff700 1 -- 172.21.15.146:0/1975424199 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66a0050450 con 0x7f6694004810 2024-08-15T17:02:51.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 msgr2=0x7f6698062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:51.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 0x7f6698062d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f669c0099c0 tx=0x7f669c01a000 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 msgr2=0x7f66ac0b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f66ac0b5be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0d0280 tx=0x7f66a0009040 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 shutdown_connections 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f6698060890 0x7f6698062d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ac157370 0x7f66ac1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ac0c7d20 0x7f66ac154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 --2- 172.21.15.146:0/1975424199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6694004810 0x7f66ac0b5be0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 >> 172.21.15.146:0/1975424199 conn(0x7f66ac099550 msgr2=0x7f66ac0ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 shutdown_connections 2024-08-15T17:02:51.363 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.346+0000 7f66b25a3700 1 -- 172.21.15.146:0/1975424199 wait complete. 2024-08-15T17:02:51.364 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_metadata' removed 2024-08-15T17:02:51.379 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-15T17:02:51.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.478+0000 7fa16a132700 1 Processor -- start 2024-08-15T17:02:51.494 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.478+0000 7fa16a132700 1 -- start start 2024-08-15T17:02:51.496 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa16415a6a0 0x7fa1640686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa16406ea10 0x7fa164068be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1640ae910 0x7fa1640699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16406c2f0 con 0x7fa16415a6a0 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640ca6b0 con 0x7fa16406ea10 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16406ba50 con 0x7fa1640ae910 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa16415a6a0 0x7fa1640686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fa16415a6a0 0x7fa1640686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59528/0 (socket says 172.21.15.146:59528) 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 -- 172.21.15.146:0/514694349 learned_addr learned my addr 172.21.15.146:0/514694349 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa163fff700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1640ae910 0x7fa1640699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa162ffd700 1 --2- 172.21.15.146:0/514694349 >> v2:172.21.15.161:3300/0 conn(0x7fa16406ea10 0x7fa164068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.497 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 -- 172.21.15.146:0/514694349 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1640ae910 msgr2=0x7fa1640699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1640ae910 0x7fa1640699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 -- 172.21.15.146:0/514694349 >> v2:172.21.15.161:3300/0 conn(0x7fa16406ea10 msgr2=0x7fa164068be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 --2- 172.21.15.146:0/514694349 >> v2:172.21.15.161:3300/0 conn(0x7fa16406ea10 0x7fa164068be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 -- 172.21.15.146:0/514694349 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16406bd30 con 0x7fa16415a6a0 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1637fe700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16415a6a0 0x7fa1640686a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa164070d90 tx=0x7fa158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ac0c4ec21dcfe server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1627fc700 1 -- 172.21.15.146:0/514694349 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800b640 con 0x7fa16415a6a0 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1627fc700 1 -- 172.21.15.146:0/514694349 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15800b7a0 con 0x7fa16415a6a0 2024-08-15T17:02:51.498 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa1627fc700 1 -- 172.21.15.146:0/514694349 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15800bb60 con 0x7fa16415a6a0 2024-08-15T17:02:51.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- 172.21.15.146:0/514694349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16415a6a0 msgr2=0x7fa1640686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:51.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16415a6a0 0x7fa1640686a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa164070d90 tx=0x7fa158009250 comp rx=0 tx=0).stop 2024-08-15T17:02:51.499 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- 172.21.15.146:0/514694349 shutdown_connections 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa1640ae910 0x7fa1640699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- 172.21.15.146:0/514694349 >> v2:172.21.15.161:3300/0 conn(0x7fa16406ea10 0x7fa164068be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 --2- 172.21.15.146:0/514694349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16415a6a0 0x7fa1640686a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- 172.21.15.146:0/514694349 >> 172.21.15.146:0/514694349 conn(0x7fa16406d9f0 msgr2=0x7fa16406ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- 172.21.15.146:0/514694349 shutdown_connections 2024-08-15T17:02:51.500 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.482+0000 7fa16a132700 1 -- 172.21.15.146:0/514694349 wait complete. 2024-08-15T17:02:51.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.486+0000 7fa16a132700 1 Processor -- start 2024-08-15T17:02:51.501 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.486+0000 7fa16a132700 1 -- start start 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1640ae910 0x7fa164151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 0x7fa164142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164152f50 con 0x7fa16406ea10 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16414f7f0 con 0x7fa1640ae910 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641522d0 con 0x7fa16415a6a0 2024-08-15T17:02:51.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa1637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa1637fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59544/0 (socket says 172.21.15.146:59544) 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa1637fe700 1 -- 172.21.15.146:0/3452423516 learned_addr learned my addr 172.21.15.146:0/3452423516 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 0x7fa164142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa162ffd700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1640ae910 0x7fa164151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 -- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1640ae910 msgr2=0x7fa164151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1640ae910 0x7fa164151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 -- 172.21.15.146:0/3452423516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 msgr2=0x7fa16415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 -- 172.21.15.146:0/3452423516 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580094f0 con 0x7fa16415a6a0 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa1637fe700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa163fff700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 0x7fa164142280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00ecc0 tx=0x7fa15c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:51.506 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00c8c0 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00f040 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1641525b0 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00cc90 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.490+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa164152b70 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1640638f0 con 0x7fa16415a6a0 2024-08-15T17:02:51.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fa15c014690 con 0x7fa16415a6a0 2024-08-15T17:02:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa14b7fe700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 0x7fa150062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:51.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6251+0+0 (secure 0 0 0) 0x7fa15c07d7b0 con 0x7fa16415a6a0 2024-08-15T17:02:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa1637fe700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 0x7fa150062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:51.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.494+0000 7fa1637fe700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 0x7fa150062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa158003080 tx=0x7fa158017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:51.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.506+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fa15c04d9e0 con 0x7fa16415a6a0 2024-08-15T17:02:51.684 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:51.670+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa164069e20 con 0x7fa16415a6a0 2024-08-15T17:02:52.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.362+0000 7fa14b7fe700 1 -- 172.21.15.146:0/3452423516 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa15c051000 con 0x7fa16415a6a0 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 msgr2=0x7fa150062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 0x7fa150062d10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa158003080 tx=0x7fa158017000 comp rx=0 tx=0).stop 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 msgr2=0x7fa164142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 0x7fa164142280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00ecc0 tx=0x7fa15c00b040 comp rx=0 tx=0).stop 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 shutdown_connections 2024-08-15T17:02:52.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fa150060860 0x7fa150062d10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa16415a6a0 0x7fa164142280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa1640ae910 0x7fa164151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 --2- 172.21.15.146:0/3452423516 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa16406ea10 0x7fa16415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 >> 172.21.15.146:0/3452423516 conn(0x7fa16406d9f0 msgr2=0x7fa1640ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 shutdown_connections 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.366+0000 7fa16a132700 1 -- 172.21.15.146:0/3452423516 wait complete. 2024-08-15T17:02:52.382 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs_data' removed 2024-08-15T17:02:52.397 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:52.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.490+0000 7f7a6cc22700 1 Processor -- start 2024-08-15T17:02:52.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.490+0000 7f7a6cc22700 1 -- start start 2024-08-15T17:02:52.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7a680687c0 0x7f7a68164170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6815d830 con 0x7f7a680c9dd0 2024-08-15T17:02:52.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6815cf00 con 0x7f7a6815a0f0 2024-08-15T17:02:52.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6815d1e0 con 0x7f7a680687c0 2024-08-15T17:02:52.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7a680c9dd0 0x7f7a680680b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49666/0 (socket says 172.21.15.146:49666) 2024-08-15T17:02:52.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 -- 172.21.15.146:0/1341770501 learned_addr learned my addr 172.21.15.146:0/1341770501 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:52.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a67fff700 1 --2- 172.21.15.146:0/1341770501 >> v2:172.21.15.161:3300/0 conn(0x7f7a680687c0 0x7f7a68164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584975401 0 0) 0x7f7a6815d830 con 0x7f7a680c9dd0 2024-08-15T17:02:52.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a50003570 con 0x7f7a680c9dd0 2024-08-15T17:02:52.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1333457907 0 0) 0x7f7a50003570 con 0x7f7a680c9dd0 2024-08-15T17:02:52.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 >> v2:172.21.15.161:3300/0 conn(0x7f7a680687c0 msgr2=0x7f7a68164170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 --2- 172.21.15.146:0/1341770501 >> v2:172.21.15.161:3300/0 conn(0x7f7a680687c0 0x7f7a68164170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 >> v1:172.21.15.161:6789/0 conn(0x7f7a6815a0f0 legacy=0x7f7a680c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6815d4c0 con 0x7f7a680c9dd0 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f7a580017f0 con 0x7f7a680c9dd0 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 >> v1:172.21.15.146:6789/0 conn(0x7f7a680c9dd0 legacy=0x7f7a680680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003a60 0x7f7a50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6815d1e0 con 0x7f7a50004810 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5000c160 con 0x7f7a500051c0 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5000c320 con 0x7f7a50003a60 2024-08-15T17:02:52.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a677fe700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003a60 0x7f7a50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a67fff700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 -- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003a60 msgr2=0x7f7a50003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a50003a60 0x7f7a50003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 -- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a500051c0 msgr2=0x7f7a5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a500051c0 0x7f7a5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 -- 172.21.15.146:0/1341770501 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50003570 con 0x7f7a50004810 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a67fff700 1 -- 172.21.15.146:0/1341770501 reap_dead start 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a66ffd700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 0x7f7a50004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800a900 tx=0x7f7a580021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97aa67f80a7aabb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a580070d0 con 0x7f7a50004810 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a58007250 con 0x7f7a50004810 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a667fc700 1 -- 172.21.15.146:0/1341770501 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007610 con 0x7f7a50004810 2024-08-15T17:02:52.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 msgr2=0x7f7a50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 0x7f7a50004c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800a900 tx=0x7f7a580021d0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- 172.21.15.146:0/1341770501 shutdown_connections 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/1341770501 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a50004810 0x7f7a50004c80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- 172.21.15.146:0/1341770501 >> 172.21.15.146:0/1341770501 conn(0x7f7a680ccfc0 msgr2=0x7f7a680c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- 172.21.15.146:0/1341770501 shutdown_connections 2024-08-15T17:02:52.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.494+0000 7f7a6cc22700 1 -- 172.21.15.146:0/1341770501 wait complete. 2024-08-15T17:02:52.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 Processor -- start 2024-08-15T17:02:52.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 -- start start 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50004810 0x7f7a68150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a680ac420 0x7f7a680adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ae590 con 0x7f7a680ae140 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68151410 con 0x7f7a50004810 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a6cc22700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68169660 con 0x7f7a680ac420 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59564/0 (socket says 172.21.15.146:59564) 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a67fff700 1 -- 172.21.15.146:0/4263176123 learned_addr learned my addr 172.21.15.146:0/4263176123 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a677fe700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50004810 0x7f7a68150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.498+0000 7f7a66ffd700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a680ac420 0x7f7a680adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 -- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a680ac420 msgr2=0x7f7a680adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a680ac420 0x7f7a680adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 -- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50004810 msgr2=0x7f7a68150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50004810 0x7f7a68150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 -- 172.21.15.146:0/4263176123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68169940 con 0x7f7a680ae140 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a67fff700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b2730 tx=0x7f7a60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60015670 con 0x7f7a680ae140 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a600157f0 con 0x7f7a680ae140 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68169c20 con 0x7f7a680ae140 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60015bb0 con 0x7f7a680ae140 2024-08-15T17:02:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6816a1e0 con 0x7f7a680ae140 2024-08-15T17:02:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.502+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6816a400 con 0x7f7a680ae140 2024-08-15T17:02:52.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.506+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f7a600143e0 con 0x7f7a680ae140 2024-08-15T17:02:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.506+0000 7f7a4f7fe700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 0x7f7a54062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.506+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f7a6007d350 con 0x7f7a680ae140 2024-08-15T17:02:52.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.506+0000 7f7a677fe700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 0x7f7a54062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.506+0000 7f7a677fe700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 0x7f7a54062d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.524 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.510+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f7a6004d7a0 con 0x7f7a680ae140 2024-08-15T17:02:52.693 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.678+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a680638f0 con 0x7f7a680ae140 2024-08-15T17:02:52.694 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.678+0000 7f7a4f7fe700 1 -- 172.21.15.146:0/4263176123 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15562 (secure 0 0 0) 0x7f7a60050dc0 con 0x7f7a680ae140 2024-08-15T17:02:52.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:52.695 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:02:52.345690+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6837/2559773181":"2024-08-16T17:02:49.326606+0000","172.21.15.161:6836/2559773181":"2024-08-16T17:02:49.326606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:52.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 msgr2=0x7f7a54062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 0x7f7a54062d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c007fd0 tx=0x7f7a5c009040 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 msgr2=0x7f7a680b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b2730 tx=0x7f7a60008040 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 shutdown_connections 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7a54060890 0x7f7a54062d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a680ae140 0x7f7a680b0720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7a680ac420 0x7f7a680adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 --2- 172.21.15.146:0/4263176123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a50004810 0x7f7a68150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 >> 172.21.15.146:0/4263176123 conn(0x7f7a680ccfc0 msgr2=0x7f7a68152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 shutdown_connections 2024-08-15T17:02:52.698 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.682+0000 7f7a6cc22700 1 -- 172.21.15.146:0/4263176123 wait complete. 2024-08-15T17:02:52.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-15T16:59:35.707086+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000940580.meta', 'create_time': '2024-08-15T17:00:41.772696+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000940580'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000940580.data', 'create_time': '2024-08-15T17:00:41.868946+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000940580'}}}] 2024-08-15T17:02:52.710 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-15T17:02:52.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.806+0000 7f237359e700 1 Processor -- start 2024-08-15T17:02:52.823 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.806+0000 7f237359e700 1 -- start start 2024-08-15T17:02:52.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f237406ead0 0x7f237406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23740b1830 0x7f2374155c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740ace30 con 0x7f237406e010 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740ae6b0 con 0x7f237406ead0 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2374156dd0 con 0x7f23740b1830 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f237406ead0 0x7f237406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f237406ead0 0x7f237406eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59568/0 (socket says 172.21.15.146:59568) 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 -- 172.21.15.146:0/2748979943 learned_addr learned my addr 172.21.15.146:0/2748979943 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f2369d9b700 1 --2- 172.21.15.146:0/2748979943 >> v2:172.21.15.161:3300/0 conn(0x7f23740b1830 0x7f2374155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f2371d9b700 1 -- 172.21.15.146:0/2748979943 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371965447 0 0) 0x7f23740ace30 con 0x7f237406e010 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 -- 172.21.15.146:0/2748979943 >> v2:172.21.15.161:3300/0 conn(0x7f23740b1830 msgr2=0x7f2374155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 --2- 172.21.15.146:0/2748979943 >> v2:172.21.15.161:3300/0 conn(0x7f23740b1830 0x7f2374155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 -- 172.21.15.146:0/2748979943 >> v1:172.21.15.161:6789/0 conn(0x7f237406e010 legacy=0x7f2374152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 -- 172.21.15.146:0/2748979943 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374156450 con 0x7f237406ead0 2024-08-15T17:02:52.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237259c700 1 --2- 172.21.15.146:0/2748979943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f237406eea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f237405f0b0 tx=0x7f2364009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76d8bf2b44ae8a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f2371d9b700 1 -- 172.21.15.146:0/2748979943 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364009e70 con 0x7f237406ead0 2024-08-15T17:02:52.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f2371d9b700 1 -- 172.21.15.146:0/2748979943 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400b860 con 0x7f237406ead0 2024-08-15T17:02:52.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f2371d9b700 1 -- 172.21.15.146:0/2748979943 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236400bc00 con 0x7f237406ead0 2024-08-15T17:02:52.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- 172.21.15.146:0/2748979943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 msgr2=0x7f237406eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 --2- 172.21.15.146:0/2748979943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f237406eea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f237405f0b0 tx=0x7f2364009250 comp rx=0 tx=0).stop 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- 172.21.15.146:0/2748979943 shutdown_connections 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 --2- 172.21.15.146:0/2748979943 >> v2:172.21.15.161:3300/0 conn(0x7f23740b1830 0x7f2374155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 --2- 172.21.15.146:0/2748979943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f237406eea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- 172.21.15.146:0/2748979943 >> 172.21.15.146:0/2748979943 conn(0x7f23740678a0 msgr2=0x7f2374067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- 172.21.15.146:0/2748979943 shutdown_connections 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.810+0000 7f237359e700 1 -- 172.21.15.146:0/2748979943 wait complete. 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 Processor -- start 2024-08-15T17:02:52.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- start start 2024-08-15T17:02:52.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 0x7f23740c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740b1830 0x7f23740aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740a9860 con 0x7f237406ead0 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740a9d50 con 0x7f23740b1830 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2374153b30 con 0x7f237406e010 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59582/0 (socket says 172.21.15.146:59582) 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 -- 172.21.15.146:0/9807874 learned_addr learned my addr 172.21.15.146:0/9807874 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237259c700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 0x7f23740c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2372d9d700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740b1830 0x7f23740aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 -- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 msgr2=0x7f23740c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 0x7f23740c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 -- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740b1830 msgr2=0x7f23740aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:52.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740b1830 0x7f23740aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 -- 172.21.15.146:0/9807874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640094f0 con 0x7f237406ead0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237259c700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 0x7f23740c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f2369d9b700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f235c00ec80 tx=0x7f235c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00c850 con 0x7f237406ead0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00c9b0 con 0x7f237406ead0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740c0fa0 con 0x7f237406ead0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00cd50 con 0x7f237406ead0 2024-08-15T17:02:52.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.814+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741537c0 con 0x7f237406ead0 2024-08-15T17:02:52.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374153ca0 con 0x7f237406ead0 2024-08-15T17:02:52.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f235c013680 con 0x7f237406ead0 2024-08-15T17:02:52.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f236affd700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 0x7f2358062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:52.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f235c07d1c0 con 0x7f237406ead0 2024-08-15T17:02:52.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f237259c700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 0x7f2358062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:52.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.818+0000 7f237259c700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 0x7f2358062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f237405f0b0 tx=0x7f2364017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:52.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.826+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f235c04d590 con 0x7f237406ead0 2024-08-15T17:02:53.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2374064330 con 0x7f237406ead0 2024-08-15T17:02:53.005 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f236affd700 1 -- 172.21.15.146:0/9807874 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f235c050bb0 con 0x7f237406ead0 2024-08-15T17:02:53.005 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 1 2024-08-15T17:02:53.007 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 msgr2=0x7f2358062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 0x7f2358062d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f237405f0b0 tx=0x7f2364017000 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 msgr2=0x7f23740aa060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f235c00ec80 tx=0x7f235c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 shutdown_connections 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2358060860 0x7f2358062d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23740b1830 0x7f23740aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f237406ead0 0x7f23740aa060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.990+0000 7f237359e700 1 --2- 172.21.15.146:0/9807874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f237406e010 0x7f23740c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.994+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 >> 172.21.15.146:0/9807874 conn(0x7f23740678a0 msgr2=0x7f23740af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.994+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 shutdown_connections 2024-08-15T17:02:53.008 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:52.994+0000 7f237359e700 1 -- 172.21.15.146:0/9807874 wait complete. 2024-08-15T17:02:53.019 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000940580.meta pg_num 2024-08-15T17:02:53.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.118+0000 7f02758cc700 1 Processor -- start 2024-08-15T17:02:53.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.118+0000 7f02758cc700 1 -- start start 2024-08-15T17:02:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 0x7f02700b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f0270153a60 0x7f02700b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027015a700 con 0x7f0270153690 2024-08-15T17:02:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02701598e0 con 0x7f0270153a60 2024-08-15T17:02:53.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0270159af0 con 0x7f027006c910 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02748ca700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f0270153690 0x7f027006c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49676/0 (socket says 172.21.15.146:49676) 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02748ca700 1 -- 172.21.15.146:0/201651915 learned_addr learned my addr 172.21.15.146:0/201651915 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02750cb700 1 --2- 172.21.15.146:0/201651915 >> v2:172.21.15.146:3300/0 conn(0x7f0270153a60 0x7f02700b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 --2- 172.21.15.146:0/201651915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 0x7f02700b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958437935 0 0) 0x7f027015a700 con 0x7f0270153690 2024-08-15T17:02:53.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f025c003570 con 0x7f0270153690 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507438793 0 0) 0x7f025c003570 con 0x7f0270153690 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 -- 172.21.15.146:0/201651915 >> v2:172.21.15.146:3300/0 conn(0x7f0270153a60 msgr2=0x7f02700b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 --2- 172.21.15.146:0/201651915 >> v2:172.21.15.146:3300/0 conn(0x7f0270153a60 0x7f02700b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 -- 172.21.15.146:0/201651915 >> v1:172.21.15.146:6789/0 conn(0x7f0270153690 legacy=0x7f027006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 -- 172.21.15.146:0/201651915 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700b0f80 con 0x7f027006c910 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026ffff700 1 --2- 172.21.15.146:0/201651915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 0x7f02700b0a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02580098e0 tx=0x7f02580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba464b5e50e5f114 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0268001810 con 0x7f027006c910 2024-08-15T17:02:53.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0258011420 con 0x7f027006c910 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f026f7fe700 1 -- 172.21.15.146:0/201651915 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580117a0 con 0x7f027006c910 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- 172.21.15.146:0/201651915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 msgr2=0x7f02700b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 --2- 172.21.15.146:0/201651915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 0x7f02700b0a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02580098e0 tx=0x7f02580091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- 172.21.15.146:0/201651915 shutdown_connections 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 --2- 172.21.15.146:0/201651915 >> v2:172.21.15.146:3300/0 conn(0x7f0270153a60 0x7f02700b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 --2- 172.21.15.146:0/201651915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f027006c910 0x7f02700b0a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- 172.21.15.146:0/201651915 >> 172.21.15.146:0/201651915 conn(0x7f02700ca2b0 msgr2=0x7f027006baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- 172.21.15.146:0/201651915 shutdown_connections 2024-08-15T17:02:53.140 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.122+0000 7f02758cc700 1 -- 172.21.15.146:0/201651915 wait complete. 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 Processor -- start 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- start start 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 0x7f02700c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0270153a60 0x7f02700c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700c3320 con 0x7f027006c910 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700c2710 con 0x7f0270153a60 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700c29f0 con 0x7f0270153690 2024-08-15T17:02:53.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59604/0 (socket says 172.21.15.146:59604) 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 -- 172.21.15.146:0/820234110 learned_addr learned my addr 172.21.15.146:0/820234110 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f026ffff700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 0x7f02700c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02750cb700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0270153a60 0x7f02700c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 -- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 msgr2=0x7f02700c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 0x7f02700c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 -- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0270153a60 msgr2=0x7f02700c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0270153a60 0x7f02700c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 -- 172.21.15.146:0/820234110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0258009420 con 0x7f027006c910 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f026ffff700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 0x7f02700c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02748ca700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f026800cca0 tx=0x7f0268001660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800ce50 con 0x7f027006c910 2024-08-15T17:02:53.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0268004dc0 con 0x7f027006c910 2024-08-15T17:02:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02700c2cd0 con 0x7f027006c910 2024-08-15T17:02:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026800abb0 con 0x7f027006c910 2024-08-15T17:02:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.126+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0270159050 con 0x7f027006c910 2024-08-15T17:02:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02700638f0 con 0x7f027006c910 2024-08-15T17:02:53.145 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0268007150 con 0x7f027006c910 2024-08-15T17:02:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f026cff9700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 0x7f0250062de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f026807c800 con 0x7f027006c910 2024-08-15T17:02:53.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f026ffff700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 0x7f0250062de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.130+0000 7f026ffff700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 0x7f0250062de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f027006cd20 tx=0x7f0258016000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.138+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f026804cd30 con 0x7f027006c910 2024-08-15T17:02:53.315 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.meta", "var": "pg_num"} v 0) v1 -- 0x7f027015a380 con 0x7f027006c910 2024-08-15T17:02:53.316 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f026cff9700 1 -- 172.21.15.146:0/820234110 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.meta", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7f02680180c0 con 0x7f027006c910 2024-08-15T17:02:53.316 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 msgr2=0x7f0250062de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 0x7f0250062de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f027006cd20 tx=0x7f0258016000 comp rx=0 tx=0).stop 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 msgr2=0x7f0270158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f026800cca0 tx=0x7f0268001660 comp rx=0 tx=0).stop 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 shutdown_connections 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0250060930 0x7f0250062de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.319 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0270153a60 0x7f02700c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0270153690 0x7f02700c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 --2- 172.21.15.146:0/820234110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f027006c910 0x7f0270158b10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 >> 172.21.15.146:0/820234110 conn(0x7f02700ca2b0 msgr2=0x7f02700b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 shutdown_connections 2024-08-15T17:02:53.320 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.302+0000 7f02758cc700 1 -- 172.21.15.146:0/820234110 wait complete. 2024-08-15T17:02:53.331 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000940580.data pg_num 2024-08-15T17:02:53.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.426+0000 7f9761e2d700 1 Processor -- start 2024-08-15T17:02:53.442 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.426+0000 7f9761e2d700 1 -- start start 2024-08-15T17:02:53.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f975c1501e0 0x7f975c0ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f975c0abd30 0x7f975c0686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.445 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f975c157160 con 0x7f975c1505b0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c156550 con 0x7f975c1501e0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c156830 con 0x7f975c0abd30 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9760e2b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f975c1501e0 0x7f975c0ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f976162c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f975c1505b0 0x7f975c0ad2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49680/0 (socket says 172.21.15.146:49680) 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f976162c700 1 -- 172.21.15.146:0/3152253358 learned_addr learned my addr 172.21.15.146:0/3152253358 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 --2- 172.21.15.146:0/3152253358 >> v2:172.21.15.161:3300/0 conn(0x7f975c0abd30 0x7f975c0686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975b7fe700 1 -- 172.21.15.146:0/3152253358 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928100170 0 0) 0x7f975c157160 con 0x7f975c1505b0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975b7fe700 1 -- 172.21.15.146:0/3152253358 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9744003570 con 0x7f975c1505b0 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 -- 172.21.15.146:0/3152253358 >> v2:172.21.15.146:3300/0 conn(0x7f975c1501e0 msgr2=0x7f975c0ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:53.446 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 --2- 172.21.15.146:0/3152253358 >> v2:172.21.15.146:3300/0 conn(0x7f975c1501e0 0x7f975c0ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 -- 172.21.15.146:0/3152253358 >> v1:172.21.15.146:6789/0 conn(0x7f975c1505b0 legacy=0x7f975c0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 -- 172.21.15.146:0/3152253358 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c156b10 con 0x7f975c0abd30 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975bfff700 1 --2- 172.21.15.146:0/3152253358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c0686a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f975c1570d0 tx=0x7f974c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e77c538588cb5cd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975b7fe700 1 -- 172.21.15.146:0/3152253358 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011630 con 0x7f975c0abd30 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975b7fe700 1 -- 172.21.15.146:0/3152253358 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c0117b0 con 0x7f975c0abd30 2024-08-15T17:02:53.447 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f975b7fe700 1 -- 172.21.15.146:0/3152253358 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011b70 con 0x7f975c0abd30 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- 172.21.15.146:0/3152253358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 msgr2=0x7f975c0686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 --2- 172.21.15.146:0/3152253358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c0686a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f975c1570d0 tx=0x7f974c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- 172.21.15.146:0/3152253358 shutdown_connections 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 --2- 172.21.15.146:0/3152253358 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c0686a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 --2- 172.21.15.146:0/3152253358 >> v2:172.21.15.146:3300/0 conn(0x7f975c1501e0 0x7f975c0ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- 172.21.15.146:0/3152253358 >> 172.21.15.146:0/3152253358 conn(0x7f975c06e310 msgr2=0x7f975c06b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- 172.21.15.146:0/3152253358 shutdown_connections 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.430+0000 7f9761e2d700 1 -- 172.21.15.146:0/3152253358 wait complete. 2024-08-15T17:02:53.451 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 Processor -- start 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- start start 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c1501e0 0x7f975c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c150a10 con 0x7f975c1505b0 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c151590 con 0x7f975c0abd30 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c150c60 con 0x7f975c1501e0 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59626/0 (socket says 172.21.15.146:59626) 2024-08-15T17:02:53.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 -- 172.21.15.146:0/2029693012 learned_addr learned my addr 172.21.15.146:0/2029693012 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f975bfff700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c1501e0 0x7f975c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9760e2b700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 -- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c1501e0 msgr2=0x7f975c13be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c1501e0 0x7f975c13be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 -- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 msgr2=0x7f975c151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 -- 172.21.15.146:0/2029693012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c009420 con 0x7f975c1505b0 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9760e2b700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:02:53.453 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f976162c700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f975400cc40 tx=0x7f9754001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400a940 con 0x7f975c1505b0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975400aaa0 con 0x7f975c1505b0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c150f40 con 0x7f975c1505b0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400ae20 con 0x7f975c1505b0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c13c3a0 con 0x7f975c1505b0 2024-08-15T17:02:53.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.434+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c0638f0 con 0x7f975c1505b0 2024-08-15T17:02:53.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.438+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9754007300 con 0x7f975c1505b0 2024-08-15T17:02:53.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.438+0000 7f9758ff9700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 0x7f9748062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.438+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f975407dbe0 con 0x7f975c1505b0 2024-08-15T17:02:53.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.438+0000 7f9760e2b700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 0x7f9748062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.442+0000 7f9760e2b700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 0x7f9748062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9750006f00 tx=0x7f9750008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.446+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f975404df90 con 0x7f975c1505b0 2024-08-15T17:02:53.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.606+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.data", "var": "pg_num"} v 0) v1 -- 0x7f975c1516d0 con 0x7f975c1505b0 2024-08-15T17:02:53.621 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.606+0000 7f9758ff9700 1 -- 172.21.15.146:0/2029693012 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000940580.data", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7f97540515b0 con 0x7f975c1505b0 2024-08-15T17:02:53.622 INFO:teuthology.orchestra.run.smithi146.stdout:pg_num: 32 2024-08-15T17:02:53.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 msgr2=0x7f9748062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 0x7f9748062d10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9750006f00 tx=0x7f9750008040 comp rx=0 tx=0).stop 2024-08-15T17:02:53.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 msgr2=0x7f975c0684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f975400cc40 tx=0x7f9754001a40 comp rx=0 tx=0).stop 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 shutdown_connections 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9748060860 0x7f9748062d10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f975c1505b0 0x7f975c0684b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f975c1501e0 0x7f975c13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 --2- 172.21.15.146:0/2029693012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f975c0abd30 0x7f975c151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 >> 172.21.15.146:0/2029693012 conn(0x7f975c06e310 msgr2=0x7f975c0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 shutdown_connections 2024-08-15T17:02:53.625 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.610+0000 7f9761e2d700 1 -- 172.21.15.146:0/2029693012 wait complete. 2024-08-15T17:02:53.636 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:53.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 Processor -- start 2024-08-15T17:02:53.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 -- start start 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f71e4157070 0x7f71e40b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71e4157440 0x7f71e415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e40b1ae0 con 0x7f71e415b980 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40ad090 con 0x7f71e4157440 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e359e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b0ed0 con 0x7f71e4157070 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71e4157440 0x7f71e415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f71e4157440 0x7f71e415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59630/0 (socket says 172.21.15.146:59630) 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 -- 172.21.15.146:0/3667263652 learned_addr learned my addr 172.21.15.146:0/3667263652 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e2d9d700 1 --1- 172.21.15.146:0/3667263652 >> v1:172.21.15.146:6789/0 conn(0x7f71e415b980 0x7f71e4066e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49682/0 (socket says 172.21.15.146:49682) 2024-08-15T17:02:53.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e259c700 1 --2- 172.21.15.146:0/3667263652 >> v2:172.21.15.161:3300/0 conn(0x7f71e4157070 0x7f71e40b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 -- 172.21.15.146:0/3667263652 >> v2:172.21.15.161:3300/0 conn(0x7f71e4157070 msgr2=0x7f71e40b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 --2- 172.21.15.146:0/3667263652 >> v2:172.21.15.161:3300/0 conn(0x7f71e4157070 0x7f71e40b0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 -- 172.21.15.146:0/3667263652 >> v1:172.21.15.146:6789/0 conn(0x7f71e415b980 legacy=0x7f71e4066e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 -- 172.21.15.146:0/3667263652 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40b11b0 con 0x7f71e4157440 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e1d9b700 1 --2- 172.21.15.146:0/3667263652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e415ad40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71e406cef0 tx=0x7f71cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e8bb19f906123e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e159a700 1 -- 172.21.15.146:0/3667263652 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00b860 con 0x7f71e4157440 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e159a700 1 -- 172.21.15.146:0/3667263652 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00b9c0 con 0x7f71e4157440 2024-08-15T17:02:53.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.726+0000 7f71e159a700 1 -- 172.21.15.146:0/3667263652 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00bd60 con 0x7f71e4157440 2024-08-15T17:02:53.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- 172.21.15.146:0/3667263652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 msgr2=0x7f71e415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- 172.21.15.146:0/3667263652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e415ad40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f71e406cef0 tx=0x7f71cc009250 comp rx=0 tx=0).stop 2024-08-15T17:02:53.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- 172.21.15.146:0/3667263652 shutdown_connections 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- 172.21.15.146:0/3667263652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e415ad40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- 172.21.15.146:0/3667263652 >> v2:172.21.15.161:3300/0 conn(0x7f71e4157070 0x7f71e40b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- 172.21.15.146:0/3667263652 >> 172.21.15.146:0/3667263652 conn(0x7f71e40ccb70 msgr2=0x7f71e40a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- 172.21.15.146:0/3667263652 shutdown_connections 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- 172.21.15.146:0/3667263652 wait complete. 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 Processor -- start 2024-08-15T17:02:53.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- start start 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 0x7f71e414e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e415b980 0x7f71e4097d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e414eb40 con 0x7f71e4157440 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e414b3f0 con 0x7f71e415b980 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e359e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e414ed50 con 0x7f71e4157070 2024-08-15T17:02:53.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59642/0 (socket says 172.21.15.146:59642) 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 -- 172.21.15.146:0/164880033 learned_addr learned my addr 172.21.15.146:0/164880033 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e259c700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 0x7f71e414e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e2d9d700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e415b980 0x7f71e4097d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 -- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 msgr2=0x7f71e414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 0x7f71e414e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 -- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e415b980 msgr2=0x7f71e4097d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e415b980 0x7f71e4097d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 -- 172.21.15.146:0/164880033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0094f0 con 0x7f71e4157440 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e259c700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 0x7f71e414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.730+0000 7f71e1d9b700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71e4070820 tx=0x7f71cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc011e60 con 0x7f71e4157440 2024-08-15T17:02:53.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc01a630 con 0x7f71e4157440 2024-08-15T17:02:53.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e4096a90 con 0x7f71e4157440 2024-08-15T17:02:53.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc01a9f0 con 0x7f71e4157440 2024-08-15T17:02:53.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e4097050 con 0x7f71e4157440 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f71cc02d440 con 0x7f71e4157440 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 0x7f71c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f71cc07e7b0 con 0x7f71e4157440 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e40638f0 con 0x7f71e4157440 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71e259c700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 0x7f71c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:53.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.734+0000 7f71e259c700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 0x7f71c8062d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71d4006fd0 tx=0x7f71d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:53.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.738+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f71cc015080 con 0x7f71e4157440 2024-08-15T17:02:53.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.938+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71e40abb30 con 0x7f71e4157440 2024-08-15T17:02:53.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.938+0000 7f71da7fc700 1 -- 172.21.15.146:0/164880033 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4623 (secure 0 0 0) 0x7f71cc04eb80 con 0x7f71e4157440 2024-08-15T17:02:53.954 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:53.954 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2911946540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2911946540},{"type":"v1","addr":"172.21.15.161:6837","nonce":2911946540}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 msgr2=0x7f71c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 0x7f71c8062d10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71d4006fd0 tx=0x7f71d4008040 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 msgr2=0x7f71e4097850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f71e4070820 tx=0x7f71cc017000 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 shutdown_connections 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f71c8060860 0x7f71c8062d10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71e415b980 0x7f71e4097d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e4157440 0x7f71e4097850 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 --2- 172.21.15.146:0/164880033 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71e4157070 0x7f71e414e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 >> 172.21.15.146:0/164880033 conn(0x7f71e40ccb70 msgr2=0x7f71e40674c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 shutdown_connections 2024-08-15T17:02:53.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:53.942+0000 7f71e359e700 1 -- 172.21.15.146:0/164880033 wait complete. 2024-08-15T17:02:53.958 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 20 2024-08-15T17:02:53.968 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:54.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 Processor -- start 2024-08-15T17:02:54.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- start start 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 0x7f24f80b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24f80b12d0 0x7f24f80cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f80b1730 con 0x7f24f80cb390 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f8068790 con 0x7f24f80b12d0 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f81583f0 con 0x7f24f8154900 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24f80b12d0 0x7f24f80cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f24f80b12d0 0x7f24f80cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59644/0 (socket says 172.21.15.146:59644) 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 -- 172.21.15.146:0/3141383482 learned_addr learned my addr 172.21.15.146:0/3141383482 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fc9d1700 1 --2- 172.21.15.146:0/3141383482 >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 0x7f24f80b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 -- 172.21.15.146:0/3141383482 >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 msgr2=0x7f24f80b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 --2- 172.21.15.146:0/3141383482 >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 0x7f24f80b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 -- 172.21.15.146:0/3141383482 >> v1:172.21.15.146:6789/0 conn(0x7f24f80cb390 legacy=0x7f24f806df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 -- 172.21.15.146:0/3141383482 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80cbaa0 con 0x7f24f80b12d0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fc9d1700 1 --2- 172.21.15.146:0/3141383482 >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 0x7f24f80b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f7fff700 1 --2- 172.21.15.146:0/3141383482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f80cae50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24f8064e60 tx=0x7f24e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f83ccc64de48fec server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f77fe700 1 -- 172.21.15.146:0/3141383482 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000b860 con 0x7f24f80b12d0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f77fe700 1 -- 172.21.15.146:0/3141383482 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000b9c0 con 0x7f24f80b12d0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24f77fe700 1 -- 172.21.15.146:0/3141383482 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000bd60 con 0x7f24f80b12d0 2024-08-15T17:02:54.080 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- 172.21.15.146:0/3141383482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 msgr2=0x7f24f80cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/3141383482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f80cae50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24f8064e60 tx=0x7f24e0009250 comp rx=0 tx=0).stop 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- 172.21.15.146:0/3141383482 shutdown_connections 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/3141383482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f80cae50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/3141383482 >> v2:172.21.15.161:3300/0 conn(0x7f24f8154900 0x7f24f80b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- 172.21.15.146:0/3141383482 >> 172.21.15.146:0/3141383482 conn(0x7f24f80678d0 msgr2=0x7f24f8067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- 172.21.15.146:0/3141383482 shutdown_connections 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- 172.21.15.146:0/3141383482 wait complete. 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 Processor -- start 2024-08-15T17:02:54.081 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.062+0000 7f24fd9d3700 1 -- start start 2024-08-15T17:02:54.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24f80cb390 0x7f24f8155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.083 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24f8154900 0x7f24f81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80aea80 con 0x7f24f80b12d0 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80af600 con 0x7f24f8154900 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80aecd0 con 0x7f24f80cb390 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59648/0 (socket says 172.21.15.146:59648) 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 -- 172.21.15.146:0/1547660395 learned_addr learned my addr 172.21.15.146:0/1547660395 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24f7fff700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24f80cb390 0x7f24f8155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd1d2700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24f8154900 0x7f24f81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 -- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24f80cb390 msgr2=0x7f24f8155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24f80cb390 0x7f24f8155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 -- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24f8154900 msgr2=0x7f24f81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24f8154900 0x7f24f81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.084 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 -- 172.21.15.146:0/1547660395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00094f0 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fc9d1700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24e8010be0 tx=0x7f24e800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800e830 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800e990 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80aefb0 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800ed50 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.066+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f8142e60 con 0x7f24f80b12d0 2024-08-15T17:02:54.085 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f80643a0 con 0x7f24f80b12d0 2024-08-15T17:02:54.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f24e8016630 con 0x7f24f80b12d0 2024-08-15T17:02:54.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24f4ff9700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 0x7f24dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f24e8081180 con 0x7f24f80b12d0 2024-08-15T17:02:54.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24f7fff700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 0x7f24dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.070+0000 7f24f7fff700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 0x7f24dc062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24f8064e60 tx=0x7f24e0017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.074+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f24e8051550 con 0x7f24f80b12d0 2024-08-15T17:02:54.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.234+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f806f5f0 con 0x7f24f80b12d0 2024-08-15T17:02:54.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.238+0000 7f24f4ff9700 1 -- 172.21.15.146:0/1547660395 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15562 (secure 0 0 0) 0x7f24e8054b70 con 0x7f24f80b12d0 2024-08-15T17:02:54.253 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:54.253 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:02:52.345690+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6837/2559773181":"2024-08-16T17:02:49.326606+0000","172.21.15.161:6836/2559773181":"2024-08-16T17:02:49.326606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:54.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 msgr2=0x7f24dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 0x7f24dc062d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24f8064e60 tx=0x7f24e0017000 comp rx=0 tx=0).stop 2024-08-15T17:02:54.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 msgr2=0x7f24f8142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f24e8010be0 tx=0x7f24e800b760 comp rx=0 tx=0).stop 2024-08-15T17:02:54.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 shutdown_connections 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24dc060860 0x7f24dc062d10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24f8154900 0x7f24f81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24f80cb390 0x7f24f8155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 --2- 172.21.15.146:0/1547660395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f24f80b12d0 0x7f24f8142920 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 >> 172.21.15.146:0/1547660395 conn(0x7f24f80678d0 msgr2=0x7f24f80c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 shutdown_connections 2024-08-15T17:02:54.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.242+0000 7f24fd9d3700 1 -- 172.21.15.146:0/1547660395 wait complete. 2024-08-15T17:02:54.271 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000940580 and related pools 2024-08-15T17:02:54.272 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:54.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.362+0000 7fad50da2700 1 Processor -- start 2024-08-15T17:02:54.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.362+0000 7fad50da2700 1 -- start start 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.362+0000 7fad50da2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad4c0ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.362+0000 7fad50da2700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad4c0cb240 0x7fad4c0cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.362+0000 7fad50da2700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c0cef40 con 0x7fad4c159340 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4c0ad930 con 0x7fad4c0cb240 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0af770 con 0x7fad4c0cddc0 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4a59c700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fad4c159340 0x7fad4c0cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49702/0 (socket says 172.21.15.146:49702) 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4a59c700 1 -- 172.21.15.146:0/3776558385 learned_addr learned my addr 172.21.15.146:0/3776558385 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad43fff700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad4c0ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> v2:172.21.15.161:3300/0 conn(0x7fad4c0cb240 0x7fad4c0cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609971176 0 0) 0x7fad4c0cef40 con 0x7fad4c159340 2024-08-15T17:02:54.383 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad38003570 con 0x7fad4c159340 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2699273969 0 0) 0x7fad38003570 con 0x7fad4c159340 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 msgr2=0x7fad4c0ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad4c0ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 >> v2:172.21.15.161:3300/0 conn(0x7fad4c0cb240 msgr2=0x7fad4c0cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 --2- 172.21.15.146:0/3776558385 >> v2:172.21.15.161:3300/0 conn(0x7fad4c0cb240 0x7fad4c0cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0ce330 con 0x7fad4c159340 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7fad3c001470 con 0x7fad4c159340 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> v2:172.21.15.161:3300/0 conn(0x7fad4c0cb240 0x7fad4c0cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 >> v1:172.21.15.146:6789/0 conn(0x7fad4c159340 legacy=0x7fad4c0cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38003a90 0x7fad38003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.384 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad38004d50 0x7fad380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0ad930 con 0x7fad38004d50 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0af770 con 0x7fad38003a90 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3800ba60 con 0x7fad4c0cddc0 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad38004d50 0x7fad380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4a59c700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38003a90 0x7fad38003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad43fff700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 -- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 msgr2=0x7fad38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:54.385 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 -- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38003a90 msgr2=0x7fad38003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38003a90 0x7fad38003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 -- 172.21.15.146:0/3776558385 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38003570 con 0x7fad38004d50 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad43fff700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0cddc0 0x7fad38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4a59c700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38003a90 0x7fad38003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:54.386 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad38004d50 0x7fad380051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad4400e3c0 tx=0x7fad4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231dfcf3ff1ca1da server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400ec10 con 0x7fad38004d50 2024-08-15T17:02:54.387 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400f040 con 0x7fad38004d50 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad49d9b700 1 -- 172.21.15.146:0/3776558385 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44014480 con 0x7fad38004d50 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- 172.21.15.146:0/3776558385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad38004d50 msgr2=0x7fad380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 --2- 172.21.15.146:0/3776558385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad38004d50 0x7fad380051c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad4400e3c0 tx=0x7fad4400b040 comp rx=0 tx=0).stop 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad4ad9d700 1 -- 172.21.15.146:0/3776558385 reap_dead start 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- 172.21.15.146:0/3776558385 shutdown_connections 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- 172.21.15.146:0/3776558385 >> 172.21.15.146:0/3776558385 conn(0x7fad4c0994e0 msgr2=0x7fad4c0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- 172.21.15.146:0/3776558385 shutdown_connections 2024-08-15T17:02:54.388 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.366+0000 7fad50da2700 1 -- 172.21.15.146:0/3776558385 wait complete. 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 Processor -- start 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- start start 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38004d50 0x7fad4c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0af8f0 0x7fad4c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b4380 con 0x7fad4c0b5510 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b5ea0 con 0x7fad38004d50 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c15d080 con 0x7fad4c0af8f0 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59664/0 (socket says 172.21.15.146:59664) 2024-08-15T17:02:54.392 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 -- 172.21.15.146:0/2811070102 learned_addr learned my addr 172.21.15.146:0/2811070102 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad4ad9d700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0af8f0 0x7fad4c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38004d50 0x7fad4c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 -- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0af8f0 msgr2=0x7fad4c0afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0af8f0 0x7fad4c0afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 -- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38004d50 msgr2=0x7fad4c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38004d50 0x7fad4c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 -- 172.21.15.146:0/2811070102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c15d360 con 0x7fad4c0b5510 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad43fff700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fad34011960 tx=0x7fad34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34015670 con 0x7fad4c0b5510 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad340157d0 con 0x7fad4c0b5510 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c15d640 con 0x7fad4c0b5510 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34015b50 con 0x7fad4c0b5510 2024-08-15T17:02:54.393 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.370+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c0b0280 con 0x7fad4c0b5510 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4c0645c0 con 0x7fad4c0b5510 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fad34013410 con 0x7fad4c0b5510 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad427fc700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fad3407d300 con 0x7fad4c0b5510 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.374+0000 7fad4a59c700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000fd0 tx=0x7fad3c007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.394 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.382+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fad3404d6d0 con 0x7fad4c0b5510 2024-08-15T17:02:54.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.574+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad4c0af070 con 0x7fad4c0b5510 2024-08-15T17:02:54.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad427fc700 1 -- 172.21.15.146:0/2811070102 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4623 (secure 0 0 0) 0x7fad34050cf0 con 0x7fad4c0b5510 2024-08-15T17:02:54.596 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:54.596 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2911946540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2911946540},{"type":"v1","addr":"172.21.15.161:6837","nonce":2911946540}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:54.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 msgr2=0x7fad2c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000fd0 tx=0x7fad3c007040 comp rx=0 tx=0).stop 2024-08-15T17:02:54.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 msgr2=0x7fad4c0b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fad34011960 tx=0x7fad34008040 comp rx=0 tx=0).stop 2024-08-15T17:02:54.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 shutdown_connections 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fad2c060860 0x7fad2c062d10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad4c0af8f0 0x7fad4c0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad4c0b5510 0x7fad4c0b5960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 --2- 172.21.15.146:0/2811070102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad38004d50 0x7fad4c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 >> 172.21.15.146:0/2811070102 conn(0x7fad4c0994e0 msgr2=0x7fad4c0ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 shutdown_connections 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.578+0000 7fad50da2700 1 -- 172.21.15.146:0/2811070102 wait complete. 2024-08-15T17:02:54.598 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 20 2024-08-15T17:02:54.607 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:02:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 Processor -- start 2024-08-15T17:02:54.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 -- start start 2024-08-15T17:02:54.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5b0406e620 0x7f5b040ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040af2b0 con 0x7f5b04155d20 2024-08-15T17:02:54.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040af590 con 0x7f5b040b0350 2024-08-15T17:02:54.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.698+0000 7f5b0aa74700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b040b07a0 con 0x7f5b0406e620 2024-08-15T17:02:54.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b09a72700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f5b04155d20 0x7f5b040aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49704/0 (socket says 172.21.15.146:49704) 2024-08-15T17:02:54.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b09a72700 1 -- 172.21.15.146:0/2122190356 learned_addr learned my addr 172.21.15.146:0/2122190356 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:54.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0a273700 1 --2- 172.21.15.146:0/2122190356 >> v2:172.21.15.161:3300/0 conn(0x7f5b0406e620 0x7f5b040ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b08a70700 1 -- 172.21.15.146:0/2122190356 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027220672 0 0) 0x7f5b040af2b0 con 0x7f5b04155d20 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b08a70700 1 -- 172.21.15.146:0/2122190356 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5aec003570 con 0x7f5b04155d20 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0a273700 1 -- 172.21.15.146:0/2122190356 >> v1:172.21.15.161:6789/0 conn(0x7f5b040b0350 legacy=0x7f5b0406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0a273700 1 -- 172.21.15.146:0/2122190356 >> v1:172.21.15.146:6789/0 conn(0x7f5b04155d20 legacy=0x7f5b040aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0a273700 1 -- 172.21.15.146:0/2122190356 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040af870 con 0x7f5b0406e620 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0a273700 1 --2- 172.21.15.146:0/2122190356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b040ad7c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0098e0 tx=0x7f5afc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad840742b7efed6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b08a70700 1 -- 172.21.15.146:0/2122190356 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00b640 con 0x7f5b0406e620 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b08a70700 1 -- 172.21.15.146:0/2122190356 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc00b7a0 con 0x7f5b0406e620 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b08a70700 1 -- 172.21.15.146:0/2122190356 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00bb40 con 0x7f5b0406e620 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- 172.21.15.146:0/2122190356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 msgr2=0x7f5b040ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/2122190356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b040ad7c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5afc0098e0 tx=0x7f5afc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- 172.21.15.146:0/2122190356 shutdown_connections 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/2122190356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b040ad7c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- 172.21.15.146:0/2122190356 >> 172.21.15.146:0/2122190356 conn(0x7f5b040678d0 msgr2=0x7f5b04067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- 172.21.15.146:0/2122190356 shutdown_connections 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- 172.21.15.146:0/2122190356 wait complete. 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 Processor -- start 2024-08-15T17:02:54.719 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.702+0000 7f5b0aa74700 1 -- start start 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b04168780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b04155d20 0x7f5b04157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0414a180 con 0x7f5b040b0350 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04149290 con 0x7f5b0406e620 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b04149570 con 0x7f5b04155d20 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59674/0 (socket says 172.21.15.146:59674) 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 -- 172.21.15.146:0/3801317916 learned_addr learned my addr 172.21.15.146:0/3801317916 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:54.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0a273700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b04155d20 0x7f5b04157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09a72700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b04168780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 -- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b04155d20 msgr2=0x7f5b04157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b04155d20 0x7f5b04157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 -- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 msgr2=0x7f5b04168780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b04168780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 -- 172.21.15.146:0/3801317916 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009420 con 0x7f5b040b0350 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09a72700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b04168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b09271700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000cb10 tx=0x7f5b000017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000a760 con 0x7f5b040b0350 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0000a8e0 con 0x7f5b040b0350 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b04149850 con 0x7f5b040b0350 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000aca0 con 0x7f5b040b0350 2024-08-15T17:02:54.723 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b04149e10 con 0x7f5b040b0350 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.706+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b040643a0 con 0x7f5b040b0350 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.710+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5b00003030 con 0x7f5b040b0350 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.710+0000 7f5af9ffb700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 0x7f5af0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.710+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f5b0007cd10 con 0x7f5b040b0350 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.710+0000 7f5b09a72700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 0x7f5af0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:54.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.710+0000 7f5b09a72700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 0x7f5af0062d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af4007fd0 tx=0x7f5af4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:54.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.718+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f5b0004d0e0 con 0x7f5b040b0350 2024-08-15T17:02:54.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.914+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b040b11b0 con 0x7f5b040b0350 2024-08-15T17:02:54.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.914+0000 7f5af9ffb700 1 -- 172.21.15.146:0/3801317916 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4623 (secure 0 0 0) 0x7f5b00050700 con 0x7f5b040b0350 2024-08-15T17:02:54.931 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:54.931 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2911946540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2911946540},{"type":"v1","addr":"172.21.15.161:6837","nonce":2911946540}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-15T17:00:42.843664+0000","modified":"2024-08-15T17:00:44.844811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"c","rank":0,"incarnation":16,"state":"up:active","state_seq":7,"addr":"172.21.15.146:6835/163298469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":163298469},{"type":"v1","addr":"172.21.15.146:6835","nonce":163298469}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000940580","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 msgr2=0x7f5af0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 0x7f5af0062d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5af4007fd0 tx=0x7f5af4009040 comp rx=0 tx=0).stop 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 msgr2=0x7f5b0413da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000cb10 tx=0x7f5b000017b0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 shutdown_connections 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5af0060860 0x7f5af0062d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5b04155d20 0x7f5b04157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.933 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5b040b0350 0x7f5b0413da60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 --2- 172.21.15.146:0/3801317916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5b0406e620 0x7f5b04168780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:54.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 >> 172.21.15.146:0/3801317916 conn(0x7f5b040678d0 msgr2=0x7f5b04153340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:54.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 shutdown_connections 2024-08-15T17:02:54.934 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:54.918+0000 7f5b0aa74700 1 -- 172.21.15.146:0/3801317916 wait complete. 2024-08-15T17:02:54.934 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 20 2024-08-15T17:02:54.945 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:02:55.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 Processor -- start 2024-08-15T17:02:55.051 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- start start 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec0b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd1ec0cb560 0x7fd1ec160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0cbaa0 con 0x7fd1ec0b12d0 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec068790 con 0x7fd1ec0cb560 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1583f0 con 0x7fd1ec154900 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f1af9700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fd1ec0b12d0 0x7fd1ec0cae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49720/0 (socket says 172.21.15.146:49720) 2024-08-15T17:02:55.053 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f1af9700 1 -- 172.21.15.146:0/2029536802 learned_addr learned my addr 172.21.15.146:0/2029536802 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f22fa700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec0b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 --2- 172.21.15.146:0/2029536802 >> v2:172.21.15.161:3300/0 conn(0x7fd1ec0cb560 0x7fd1ec160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f12f8700 1 -- 172.21.15.146:0/2029536802 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584936641 0 0) 0x7fd1ec0cbaa0 con 0x7fd1ec0b12d0 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f12f8700 1 -- 172.21.15.146:0/2029536802 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d4003570 con 0x7fd1ec0b12d0 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 -- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 msgr2=0x7fd1ec0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec0b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 -- 172.21.15.146:0/2029536802 >> v1:172.21.15.146:6789/0 conn(0x7fd1ec0b12d0 legacy=0x7fd1ec0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.054 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 -- 172.21.15.146:0/2029536802 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0b1700 con 0x7fd1ec0cb560 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f2afb700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec160ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40098e0 tx=0x7fd1e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3039cd3f84bd879f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f12f8700 1 -- 172.21.15.146:0/2029536802 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4011630 con 0x7fd1ec0cb560 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f12f8700 1 -- 172.21.15.146:0/2029536802 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4011790 con 0x7fd1ec0cb560 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f12f8700 1 -- 172.21.15.146:0/2029536802 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4011b30 con 0x7fd1ec0cb560 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0cb560 msgr2=0x7fd1ec160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec160ac0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40098e0 tx=0x7fd1e40091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- 172.21.15.146:0/2029536802 shutdown_connections 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec160ac0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/2029536802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec0b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- 172.21.15.146:0/2029536802 >> 172.21.15.146:0/2029536802 conn(0x7fd1ec0678d0 msgr2=0x7fd1ec067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- 172.21.15.146:0/2029536802 shutdown_connections 2024-08-15T17:02:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.038+0000 7fd1f32fc700 1 -- 172.21.15.146:0/2029536802 wait complete. 2024-08-15T17:02:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 Processor -- start 2024-08-15T17:02:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- start start 2024-08-15T17:02:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 0x7fd1ec1692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec16a290 con 0x7fd1ec0cb560 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec150030 con 0x7fd1ec0b12d0 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec1699f0 con 0x7fd1ec154900 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59684/0 (socket says 172.21.15.146:59684) 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 -- 172.21.15.146:0/507579482 learned_addr learned my addr 172.21.15.146:0/507579482 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f2afb700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f22fa700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 0x7fd1ec1692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 -- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 msgr2=0x7fd1ec14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec14faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 -- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 msgr2=0x7fd1ec1692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 0x7fd1ec1692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 -- 172.21.15.146:0/507579482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009420 con 0x7fd1ec0cb560 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f2afb700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f22fa700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 0x7fd1ec1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f1af9700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800cc40 tx=0x7fd1e8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800a8b0 con 0x7fd1ec0cb560 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e800aa30 con 0x7fd1ec0cb560 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800adf0 con 0x7fd1ec0cb560 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec169cd0 con 0x7fd1ec0cb560 2024-08-15T17:02:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.042+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0aeee0 con 0x7fd1ec0cb560 2024-08-15T17:02:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd1e8007300 con 0x7fd1ec0cb560 2024-08-15T17:02:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec0aed40 con 0x7fd1ec0cb560 2024-08-15T17:02:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1e27fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 0x7fd1d8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fd1e807d040 con 0x7fd1ec0cb560 2024-08-15T17:02:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1f22fa700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 0x7fd1d8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.050+0000 7fd1f22fa700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 0x7fd1d8062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0ca4b0 tx=0x7fd1dc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.054+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd1ec0aed40 con 0x7fd1ec0cb560 2024-08-15T17:02:55.228 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.218+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1ec0aed40 con 0x7fd1ec0cb560 2024-08-15T17:02:55.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.218+0000 7fd1e27fc700 1 -- 172.21.15.146:0/507579482 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+15562 (secure 0 0 0) 0x7fd1ec0aed40 con 0x7fd1ec0cb560 2024-08-15T17:02:55.230 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:02:55.230 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":40,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:02:52.345690+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000940580.meta","create_time":"2024-08-15T17:00:41.772696+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000940580"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000940580.data","create_time":"2024-08-15T17:00:41.868946+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000940580"}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.161:6837/2559773181":"2024-08-16T17:02:49.326606+0000","172.21.15.161:6836/2559773181":"2024-08-16T17:02:49.326606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 msgr2=0x7fd1d8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 0x7fd1d8062d10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0ca4b0 tx=0x7fd1dc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 msgr2=0x7fd1ec154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800cc40 tx=0x7fd1e8001a70 comp rx=0 tx=0).stop 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 shutdown_connections 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd1d8060860 0x7fd1d8062d10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1ec154900 0x7fd1ec14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd1ec0cb560 0x7fd1ec154db0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 --2- 172.21.15.146:0/507579482 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1ec0b12d0 0x7fd1ec1692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 >> 172.21.15.146:0/507579482 conn(0x7fd1ec0678d0 msgr2=0x7fd1ec1518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 shutdown_connections 2024-08-15T17:02:55.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.222+0000 7fd1f32fc700 1 -- 172.21.15.146:0/507579482 wait complete. 2024-08-15T17:02:55.245 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000940580 2024-08-15T17:02:55.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.342+0000 7f70f802d700 1 Processor -- start 2024-08-15T17:02:55.354 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.342+0000 7f70f802d700 1 -- start start 2024-08-15T17:02:55.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70f0159340 0x7f70f0159710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00b1cf0 con 0x7f70f00b4330 2024-08-15T17:02:55.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f00b60d0 con 0x7f70f0159c50 2024-08-15T17:02:55.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f00b10e0 con 0x7f70f0159340 2024-08-15T17:02:55.356 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f5dc9700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f70f0159340 0x7f70f0159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f65ca700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f70f00b4330 0x7f70f00b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49736/0 (socket says 172.21.15.146:49736) 2024-08-15T17:02:55.357 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f65ca700 1 -- 172.21.15.146:0/2746052657 learned_addr learned my addr 172.21.15.146:0/2746052657 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f5dc9700 1 -- 172.21.15.146:0/2746052657 >> v1:172.21.15.161:6789/0 conn(0x7f70f0159c50 legacy=0x7f70f00b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f5dc9700 1 -- 172.21.15.146:0/2746052657 >> v1:172.21.15.146:6789/0 conn(0x7f70f00b4330 legacy=0x7f70f00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f4dc7700 1 -- 172.21.15.146:0/2746052657 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617517346 0 0) 0x7f70f00b1cf0 con 0x7f70f00b4330 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f5dc9700 1 -- 172.21.15.146:0/2746052657 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00b13c0 con 0x7f70f0159340 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f5dc9700 1 --2- 172.21.15.146:0/2746052657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0159340 0x7f70f0159710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009e60 tx=0x7f70e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9f4dc1cad71f17 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f4dc7700 1 -- 172.21.15.146:0/2746052657 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800e070 con 0x7f70f0159340 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f4dc7700 1 -- 172.21.15.146:0/2746052657 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e800bc30 con 0x7f70f0159340 2024-08-15T17:02:55.358 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f4dc7700 1 -- 172.21.15.146:0/2746052657 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8012430 con 0x7f70f0159340 2024-08-15T17:02:55.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- 172.21.15.146:0/2746052657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0159340 msgr2=0x7f70f0159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 --2- 172.21.15.146:0/2746052657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0159340 0x7f70f0159710 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009e60 tx=0x7f70e8009780 comp rx=0 tx=0).stop 2024-08-15T17:02:55.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- 172.21.15.146:0/2746052657 shutdown_connections 2024-08-15T17:02:55.359 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 --2- 172.21.15.146:0/2746052657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f0159340 0x7f70f0159710 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- 172.21.15.146:0/2746052657 >> 172.21.15.146:0/2746052657 conn(0x7f70f00994f0 msgr2=0x7f70f00675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:55.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- 172.21.15.146:0/2746052657 shutdown_connections 2024-08-15T17:02:55.360 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.346+0000 7f70f802d700 1 -- 172.21.15.146:0/2746052657 wait complete. 2024-08-15T17:02:55.361 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.350+0000 7f70f802d700 1 Processor -- start 2024-08-15T17:02:55.362 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.350+0000 7f70f802d700 1 -- start start 2024-08-15T17:02:55.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0159340 0x7f70f014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f70f0159c50 0x7f70f016ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.366 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015ed60 con 0x7f70f00b4330 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015e150 con 0x7f70f0159340 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f015e430 con 0x7f70f0159c50 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59700/0 (socket says 172.21.15.146:59700) 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 -- 172.21.15.146:0/837939744 learned_addr learned my addr 172.21.15.146:0/837939744 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f55c8700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0159340 0x7f70f014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f65ca700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f70f0159c50 0x7f70f016ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 -- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f70f0159c50 msgr2=0x7f70f016ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f70f0159c50 0x7f70f016ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 -- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0159340 msgr2=0x7f70f014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0159340 0x7f70f014e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 -- 172.21.15.146:0/837939744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e8009a20 con 0x7f70f00b4330 2024-08-15T17:02:55.367 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f5dc9700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f70e80099f0 tx=0x7f70e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e801b420 con 0x7f70f00b4330 2024-08-15T17:02:55.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e801b580 con 0x7f70f00b4330 2024-08-15T17:02:55.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f015e710 con 0x7f70f00b4330 2024-08-15T17:02:55.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e801b940 con 0x7f70f00b4330 2024-08-15T17:02:55.368 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f014f8c0 con 0x7f70f00b4330 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.354+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f00645c0 con 0x7f70f00b4330 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.358+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f70e801bcf0 con 0x7f70f00b4330 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.358+0000 7f70e5ffb700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 0x7f70d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.358+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f70e807e9a0 con 0x7f70f00b4330 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.358+0000 7f70f55c8700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 0x7f70d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.369 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.358+0000 7f70f55c8700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 0x7f70d4062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70f009d860 tx=0x7f70e0007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.375 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.362+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f70e804ecd0 con 0x7f70f00b4330 2024-08-15T17:02:55.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.566+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000940580"} v 0) v1 -- 0x7f70f00b4760 con 0x7f70f00b4330 2024-08-15T17:02:55.576 INFO:tasks.ceph.mon.a.smithi146.stderr:2024-08-15T17:02:55.566+0000 7fbf03872700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-15T17:02:55.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70e5ffb700 1 -- 172.21.15.146:0/837939744 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000940580"}]=0 volume_0000000000940580 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 195+0+0 (secure 0 0 0) 0x7f70e802a020 con 0x7f70f00b4330 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 msgr2=0x7f70d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 0x7f70d4062d10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70f009d860 tx=0x7f70e0007ec0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 msgr2=0x7f70f014f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f70e80099f0 tx=0x7f70e8019000 comp rx=0 tx=0).stop 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 shutdown_connections 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f70d4060860 0x7f70d4062d10 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f70f0159c50 0x7f70f016ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70f0159340 0x7f70f014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 --2- 172.21.15.146:0/837939744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f70f00b4330 0x7f70f014f380 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 >> 172.21.15.146:0/837939744 conn(0x7f70f00994f0 msgr2=0x7f70f015c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 shutdown_connections 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.574+0000 7f70f802d700 1 -- 172.21.15.146:0/837939744 wait complete. 2024-08-15T17:02:55.588 INFO:teuthology.orchestra.run.smithi146.stderr:volume_0000000000940580 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-15T17:02:55.599 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000940580 --yes-i-really-mean-it 2024-08-15T17:02:55.625 INFO:tasks.ceph.mds.c.smithi146.stdout:starting mds.c at 2024-08-15T17:02:55.705 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 Processor -- start 2024-08-15T17:02:55.706 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 -- start start 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 0x7fea04142a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea040b1180 con 0x7fea040c82e0 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea040b0890 con 0x7fea040ae510 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea09506700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c9310 con 0x7fea04142660 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea08d05700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fea040c82e0 0x7fea040c8760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49738/0 (socket says 172.21.15.146:49738) 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea08d05700 1 -- 172.21.15.146:0/3099721200 learned_addr learned my addr 172.21.15.146:0/3099721200 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.694+0000 7fea03fff700 1 --2- 172.21.15.146:0/3099721200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 0x7fea04142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269118587 0 0) 0x7fea040b1180 con 0x7fea040c82e0 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f0003570 con 0x7fea040c82e0 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea03fff700 1 -- 172.21.15.146:0/3099721200 >> v1:172.21.15.161:6789/0 conn(0x7fea040ae510 legacy=0x7fea040ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182589616 0 0) 0x7fea040b0890 con 0x7fea040ae510 2024-08-15T17:02:55.708 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea03fff700 1 -- 172.21.15.146:0/3099721200 >> v1:172.21.15.146:6789/0 conn(0x7fea040c82e0 legacy=0x7fea040c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea03fff700 1 -- 172.21.15.146:0/3099721200 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040c9030 con 0x7fea04142660 2024-08-15T17:02:55.709 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea03fff700 1 --2- 172.21.15.146:0/3099721200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 0x7fea04142a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e90 tx=0x7fe9fc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9e4fb4fa8ce051f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00e070 con 0x7fea04142660 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00bc30 con 0x7fea04142660 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea02ffd700 1 -- 172.21.15.146:0/3099721200 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc012430 con 0x7fea04142660 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 -- 172.21.15.146:0/3099721200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 msgr2=0x7fea04142a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 --2- 172.21.15.146:0/3099721200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 0x7fea04142a30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e90 tx=0x7fe9fc009780 comp rx=0 tx=0).stop 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 -- 172.21.15.146:0/3099721200 shutdown_connections 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 --2- 172.21.15.146:0/3099721200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea04142660 0x7fea04142a30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.710 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 -- 172.21.15.146:0/3099721200 >> 172.21.15.146:0/3099721200 conn(0x7fea040c9830 msgr2=0x7fea040c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:55.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 -- 172.21.15.146:0/3099721200 shutdown_connections 2024-08-15T17:02:55.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.698+0000 7fea09506700 1 -- 172.21.15.146:0/3099721200 wait complete. 2024-08-15T17:02:55.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 Processor -- start 2024-08-15T17:02:55.711 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- start start 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 0x7fea040c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea040c82e0 0x7fea040c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c5bd0 con 0x7fea04142660 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c4fc0 con 0x7fea040ae510 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040c52a0 con 0x7fea040c82e0 2024-08-15T17:02:55.714 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59720/0 (socket says 172.21.15.146:59720) 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 -- 172.21.15.146:0/3460658014 learned_addr learned my addr 172.21.15.146:0/3460658014 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea03fff700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 0x7fea040c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea037fe700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea040c82e0 0x7fea040c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 -- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea040c82e0 msgr2=0x7fea040c1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea040c82e0 0x7fea040c1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 -- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 msgr2=0x7fea040c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 0x7fea040c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 -- 172.21.15.146:0/3460658014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc009a20 con 0x7fea04142660 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea03fff700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 0x7fea040c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:55.715 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea08d05700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40010e0 tx=0x7fe9f4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4005c00 con 0x7fea04142660 2024-08-15T17:02:55.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4005d60 con 0x7fea04142660 2024-08-15T17:02:55.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea040c5580 con 0x7fea04142660 2024-08-15T17:02:55.716 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40072e0 con 0x7fea04142660 2024-08-15T17:02:55.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.702+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea040c4be0 con 0x7fea04142660 2024-08-15T17:02:55.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.706+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea040638f0 con 0x7fea04142660 2024-08-15T17:02:55.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.706+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe9f4012400 con 0x7fea04142660 2024-08-15T17:02:55.717 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.706+0000 7fe9fbfff700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 0x7fe9e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:55.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.706+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7fe9f407c0e0 con 0x7fea04142660 2024-08-15T17:02:55.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.706+0000 7fea03fff700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 0x7fe9e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:55.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.710+0000 7fea03fff700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 0x7fe9e4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009de0 tx=0x7fe9fc019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:55.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.714+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe9f404a480 con 0x7fea04142660 2024-08-15T17:02:55.926 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:55.914+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000940580", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea04158e40 con 0x7fea04142660 2024-08-15T17:02:56.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.582+0000 7fe9fbfff700 1 -- 172.21.15.146:0/3460658014 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000940580", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 121+0+0 (secure 0 0 0) 0x7fe9f404c470 con 0x7fea04142660 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 msgr2=0x7fe9e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 0x7fe9e4062d10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009de0 tx=0x7fe9fc019000 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 msgr2=0x7fea040c2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40010e0 tx=0x7fe9f4007dd0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 shutdown_connections 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe9e4060860 0x7fe9e4062d10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fea04142660 0x7fea040c2430 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fea040c82e0 0x7fea040c1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 --2- 172.21.15.146:0/3460658014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fea040ae510 0x7fea040c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.597 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 >> 172.21.15.146:0/3460658014 conn(0x7fea040c9830 msgr2=0x7fea04159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:56.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 shutdown_connections 2024-08-15T17:02:56.598 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.586+0000 7fea09506700 1 -- 172.21.15.146:0/3460658014 wait complete. 2024-08-15T17:02:56.612 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000940580.meta cephfs.volume_0000000000940580.meta --yes-i-really-really-mean-it 2024-08-15T17:02:56.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.766+0000 7f3100a7d700 1 Processor -- start 2024-08-15T17:02:56.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.766+0000 7f3100a7d700 1 -- start start 2024-08-15T17:02:56.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30fc0d0c20 0x7f30fc0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc0ce100 con 0x7f30fc0cead0 2024-08-15T17:02:56.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0cde60 con 0x7f30fc0d0c20 2024-08-15T17:02:56.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0cc1b0 con 0x7f30fc15c7d0 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30fc0d0c20 0x7f30fc0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30fa59c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f30fc0d0c20 0x7f30fc0ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:50350/0 (socket says 172.21.15.146:50350) 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 -- 172.21.15.146:0/2373319087 learned_addr learned my addr 172.21.15.146:0/2373319087 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 -- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 msgr2=0x7f30fc0b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:56.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 -- 172.21.15.146:0/2373319087 >> v1:172.21.15.161:6789/0 conn(0x7f30fc0cead0 legacy=0x7f30fc0c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 -- 172.21.15.146:0/2373319087 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0cc490 con 0x7f30fc0d0c20 2024-08-15T17:02:56.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30fa59c700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:56.782 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f9d9b700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0ce590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0099b0 tx=0x7f30ec009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=695dd68e79eca504 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f959a700 1 -- 172.21.15.146:0/2373319087 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec009ea0 con 0x7f30fc0d0c20 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f959a700 1 -- 172.21.15.146:0/2373319087 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec00b860 con 0x7f30fc0d0c20 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f30f959a700 1 -- 172.21.15.146:0/2373319087 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00bc00 con 0x7f30fc0d0c20 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 msgr2=0x7f30fc0ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0ce590 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0099b0 tx=0x7f30ec009250 comp rx=0 tx=0).stop 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- 172.21.15.146:0/2373319087 shutdown_connections 2024-08-15T17:02:56.783 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0ce590 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 --2- 172.21.15.146:0/2373319087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- 172.21.15.146:0/2373319087 >> 172.21.15.146:0/2373319087 conn(0x7f30fc067590 msgr2=0x7f30fc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- 172.21.15.146:0/2373319087 shutdown_connections 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.770+0000 7f3100a7d700 1 -- 172.21.15.146:0/2373319087 wait complete. 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 Processor -- start 2024-08-15T17:02:56.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 -- start start 2024-08-15T17:02:56.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c7340 con 0x7f30fc0cead0 2024-08-15T17:02:56.786 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c8160 con 0x7f30fc0d0c20 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f3100a7d700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0c85f0 con 0x7f30fc15c7d0 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59722/0 (socket says 172.21.15.146:59722) 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30f9d9b700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 -- 172.21.15.146:0/1917802650 learned_addr learned my addr 172.21.15.146:0/1917802650 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:56.787 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fad9d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 -- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 msgr2=0x7f30fc0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 -- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 msgr2=0x7f30fc0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:56.788 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 -- 172.21.15.146:0/1917802650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0009420 con 0x7f30fc0cead0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30f9d9b700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.774+0000 7f30fa59c700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f30f000ec80 tx=0x7f30f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000c920 con 0x7f30fc0cead0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ec0094f0 con 0x7f30fc0cead0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f000ca80 con 0x7f30fc0cead0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ce00 con 0x7f30fc0cead0 2024-08-15T17:02:56.789 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc0c7830 con 0x7f30fc0cead0 2024-08-15T17:02:56.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc0638f0 con 0x7f30fc0cead0 2024-08-15T17:02:56.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f30f0013680 con 0x7f30fc0cead0 2024-08-15T17:02:56.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 0x7f30dc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:56.790 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5923+0+0 (secure 0 0 0) 0x7f30f007d0e0 con 0x7f30fc0cead0 2024-08-15T17:02:56.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.778+0000 7f30f9d9b700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 0x7f30dc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:56.791 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.782+0000 7f30f9d9b700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 0x7f30dc062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0099b0 tx=0x7f30ec017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:56.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.786+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f30f004d450 con 0x7f30fc0cead0 2024-08-15T17:02:56.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:56.950+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000940580.meta", "pool2": "cephfs.volume_0000000000940580.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30fc068050 con 0x7f30fc0cead0 2024-08-15T17:02:57.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.586+0000 7f30ea7fc700 1 -- 172.21.15.146:0/1917802650 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000940580.meta", "pool2": "cephfs.volume_0000000000940580.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000940580.meta' removed v42) v1 ==== 241+0+0 (secure 0 0 0) 0x7f30f0050a70 con 0x7f30fc0cead0 2024-08-15T17:02:57.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 msgr2=0x7f30dc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 0x7f30dc062d10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0099b0 tx=0x7f30ec017000 comp rx=0 tx=0).stop 2024-08-15T17:02:57.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 msgr2=0x7f30fc14afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f30f000ec80 tx=0x7f30f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 shutdown_connections 2024-08-15T17:02:57.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f30dc060860 0x7f30dc062d10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15c7d0 0x7f30fc0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0d0c20 0x7f30fc0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 --2- 172.21.15.146:0/1917802650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30fc0cead0 0x7f30fc14afa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 >> 172.21.15.146:0/1917802650 conn(0x7f30fc067590 msgr2=0x7f30fc0ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 shutdown_connections 2024-08-15T17:02:57.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.594+0000 7f3100a7d700 1 -- 172.21.15.146:0/1917802650 wait complete. 2024-08-15T17:02:57.608 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs.volume_0000000000940580.meta' removed 2024-08-15T17:02:57.623 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000940580.data cephfs.volume_0000000000940580.data --yes-i-really-really-mean-it 2024-08-15T17:02:57.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 Processor -- start 2024-08-15T17:02:57.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 -- start start 2024-08-15T17:02:57.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7f00aec20 0x7fb7f006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:57.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f015ba90 con 0x7fb7f0068db0 2024-08-15T17:02:57.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f006a870 con 0x7fb7f00ae850 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f8343700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f006ec50 con 0x7fb7f00aec20 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f58de700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fb7f00aec20 0x7fb7f006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f68e0700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb7f0068db0 0x7fb7f0162ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:49746/0 (socket says 172.21.15.146:49746) 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f68e0700 1 -- 172.21.15.146:0/222141425 learned_addr learned my addr 172.21.15.146:0/222141425 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f58de700 1 -- 172.21.15.146:0/222141425 >> v1:172.21.15.161:6789/0 conn(0x7fb7f00ae850 legacy=0x7fb7f00686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.726+0000 7fb7f58de700 1 -- 172.21.15.146:0/222141425 >> v1:172.21.15.146:6789/0 conn(0x7fb7f0068db0 legacy=0x7fb7f0162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f58de700 1 -- 172.21.15.146:0/222141425 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f01599c0 con 0x7fb7f00aec20 2024-08-15T17:02:57.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f58de700 1 --2- 172.21.15.146:0/222141425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f006bce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00098e0 tx=0x7fb7e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d66ccc5ba80f375 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:57.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f50dd700 1 -- 172.21.15.146:0/222141425 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0009e80 con 0x7fb7f00aec20 2024-08-15T17:02:57.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f50dd700 1 -- 172.21.15.146:0/222141425 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e000b860 con 0x7fb7f00aec20 2024-08-15T17:02:57.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f50dd700 1 -- 172.21.15.146:0/222141425 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000bbe0 con 0x7fb7f00aec20 2024-08-15T17:02:57.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 -- 172.21.15.146:0/222141425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 msgr2=0x7fb7f006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 --2- 172.21.15.146:0/222141425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f006bce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb7e00098e0 tx=0x7fb7e00091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 -- 172.21.15.146:0/222141425 shutdown_connections 2024-08-15T17:02:57.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 --2- 172.21.15.146:0/222141425 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f006bce0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 -- 172.21.15.146:0/222141425 >> 172.21.15.146:0/222141425 conn(0x7fb7f006d9f0 msgr2=0x7fb7f006ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:57.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 -- 172.21.15.146:0/222141425 shutdown_connections 2024-08-15T17:02:57.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.730+0000 7fb7f8343700 1 -- 172.21.15.146:0/222141425 wait complete. 2024-08-15T17:02:57.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 Processor -- start 2024-08-15T17:02:57.745 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- start start 2024-08-15T17:02:57.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 0x7fb7f0169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:57.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7f00ae850 0x7fb7f0157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:57.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0156a90 con 0x7fb7f00aec20 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0156d70 con 0x7fb7f0068db0 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0157050 con 0x7fb7f00ae850 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:59752/0 (socket says 172.21.15.146:59752) 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 -- 172.21.15.146:0/3539517773 learned_addr learned my addr 172.21.15.146:0/3539517773 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f60df700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 0x7fb7f0169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f58de700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7f00ae850 0x7fb7f0157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 -- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7f00ae850 msgr2=0x7fb7f0157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.749 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7f00ae850 0x7fb7f0157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 -- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 msgr2=0x7fb7f0169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 0x7fb7f0169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 -- 172.21.15.146:0/3539517773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009420 con 0x7fb7f00aec20 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f60df700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 0x7fb7f0169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f68e0700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800cc70 tx=0x7fb7e8001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800ce80 con 0x7fb7f00aec20 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800ab10 con 0x7fb7f00aec20 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f0157330 con 0x7fb7f00aec20 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80070f0 con 0x7fb7f00aec20 2024-08-15T17:02:57.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.734+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f0169ff0 con 0x7fb7f00aec20 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f01538b0 con 0x7fb7f00aec20 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb7e80074e0 con 0x7fb7f00aec20 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7e67fc700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 0x7fb7d4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5448+0+0 (secure 0 0 0) 0x7fb7e807cef0 con 0x7fb7f00aec20 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7f60df700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 0x7fb7d4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:57.751 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.738+0000 7fb7f60df700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 0x7fb7d4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec007fd0 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:57.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.742+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb7f01538b0 con 0x7fb7f00aec20 2024-08-15T17:02:57.916 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:57.906+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000940580.data", "pool2": "cephfs.volume_0000000000940580.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7f01538b0 con 0x7fb7f00aec20 2024-08-15T17:02:58.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.598+0000 7fb7e67fc700 1 -- 172.21.15.146:0/3539517773 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000940580.data", "pool2": "cephfs.volume_0000000000940580.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000940580.data' removed v43) v1 ==== 241+0+0 (secure 0 0 0) 0x7fb7f01538b0 con 0x7fb7f00aec20 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 msgr2=0x7fb7d4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 0x7fb7d4062d10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec007fd0 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 msgr2=0x7fb7f0158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800cc70 tx=0x7fb7e8001a20 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 shutdown_connections 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb7d4060860 0x7fb7d4062d10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7f00aec20 0x7fb7f0158070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7f00ae850 0x7fb7f0157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 --2- 172.21.15.146:0/3539517773 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7f0068db0 0x7fb7f0169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 >> 172.21.15.146:0/3539517773 conn(0x7fb7f006d9f0 msgr2=0x7fb7f00a9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 shutdown_connections 2024-08-15T17:02:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.602+0000 7fb7f8343700 1 -- 172.21.15.146:0/3539517773 wait complete. 2024-08-15T17:02:58.614 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'cephfs.volume_0000000000940580.data' removed 2024-08-15T17:02:58.629 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-08-15T17:02:58.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.738+0000 7f250f78e700 1 Processor -- start 2024-08-15T17:02:58.750 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.738+0000 7f250f78e700 1 -- start start 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f25080cc840 0x7f25080ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080b5750 0x7f25080b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080cd150 con 0x7f25080b2210 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25080af570 con 0x7f25080cc840 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2508159ce0 con 0x7f25080b5750 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250cd29700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f25080b2210 0x7f25080b5040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41824/0 (socket says 172.21.15.146:41824) 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250cd29700 1 -- 172.21.15.146:0/211618248 learned_addr learned my addr 172.21.15.146:0/211618248 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:58.752 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 --2- 172.21.15.146:0/211618248 >> v2:172.21.15.146:3300/0 conn(0x7f25080cc840 0x7f25080ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.753 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250dd2b700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080b5750 0x7f25080b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f24fffff700 1 -- 172.21.15.146:0/211618248 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165598680 0 0) 0x7f25080cd150 con 0x7f25080b2210 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f24fffff700 1 -- 172.21.15.146:0/211618248 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f4003570 con 0x7f25080b2210 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 -- 172.21.15.146:0/211618248 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080b5750 msgr2=0x7f25080b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080b5750 0x7f25080b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 -- 172.21.15.146:0/211618248 >> v1:172.21.15.146:6789/0 conn(0x7f25080b2210 legacy=0x7f25080b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 -- 172.21.15.146:0/211618248 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2508159fc0 con 0x7f25080cc840 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250d52a700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080cc840 0x7f25080ccc10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f25040098e0 tx=0x7f25040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2419aa28117e4fe4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f24fffff700 1 -- 172.21.15.146:0/211618248 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011630 con 0x7f25080cc840 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f24fffff700 1 -- 172.21.15.146:0/211618248 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011790 con 0x7f25080cc840 2024-08-15T17:02:58.754 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f24fffff700 1 -- 172.21.15.146:0/211618248 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011b10 con 0x7f25080cc840 2024-08-15T17:02:58.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- 172.21.15.146:0/211618248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080cc840 msgr2=0x7f25080ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.755 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080cc840 0x7f25080ccc10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f25040098e0 tx=0x7f25040091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- 172.21.15.146:0/211618248 shutdown_connections 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080b5750 0x7f25080b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 --2- 172.21.15.146:0/211618248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080cc840 0x7f25080ccc10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- 172.21.15.146:0/211618248 >> 172.21.15.146:0/211618248 conn(0x7f25080682c0 msgr2=0x7f25080686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- 172.21.15.146:0/211618248 shutdown_connections 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.742+0000 7f250f78e700 1 -- 172.21.15.146:0/211618248 wait complete. 2024-08-15T17:02:58.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 Processor -- start 2024-08-15T17:02:58.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 -- start start 2024-08-15T17:02:58.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25080b5750 0x7f250815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080cc840 0x7f25080b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25080c7f10 con 0x7f25080b2210 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25080c7300 con 0x7f25080b5750 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250f78e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25080c75e0 con 0x7f25080cc840 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250d52a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250d52a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33446/0 (socket says 172.21.15.146:33446) 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250d52a700 1 -- 172.21.15.146:0/3520840810 learned_addr learned my addr 172.21.15.146:0/3520840810 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250cd29700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25080b5750 0x7f250815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.746+0000 7f250dd2b700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080cc840 0x7f25080b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 -- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080cc840 msgr2=0x7f25080b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080cc840 0x7f25080b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 -- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25080b5750 msgr2=0x7f250815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25080b5750 0x7f250815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 -- 172.21.15.146:0/3520840810 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504009420 con 0x7f25080b2210 2024-08-15T17:02:58.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250d52a700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25080884f0 tx=0x7f2504017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011900 con 0x7f25080b2210 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011a60 con 0x7f25080b2210 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25080c78c0 con 0x7f25080b2210 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250401a680 con 0x7f25080b2210 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25080c83a0 con 0x7f25080b2210 2024-08-15T17:02:58.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.750+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25080645c0 con 0x7f25080b2210 2024-08-15T17:02:58.763 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.754+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f250401aa70 con 0x7f25080b2210 2024-08-15T17:02:58.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.754+0000 7f24fd7fa700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 0x7f24ec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:58.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.754+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f250407e370 con 0x7f25080b2210 2024-08-15T17:02:58.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.754+0000 7f250cd29700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 0x7f24ec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:58.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.754+0000 7f250cd29700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 0x7f24ec062d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f250815c0d0 tx=0x7f24f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:58.771 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.758+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f250404ea60 con 0x7f25080b2210 2024-08-15T17:02:58.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.926+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f25080b1570 con 0x7f25080b2210 2024-08-15T17:02:58.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.934+0000 7f24fd7fa700 1 -- 172.21.15.146:0/3520840810 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2504052080 con 0x7f25080b2210 2024-08-15T17:02:58.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 msgr2=0x7f24ec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 0x7f24ec062d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f250815c0d0 tx=0x7f24f8009040 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 msgr2=0x7f25080c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25080884f0 tx=0x7f2504017000 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 shutdown_connections 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f24ec060890 0x7f24ec062d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25080cc840 0x7f25080b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25080b5750 0x7f250815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 --2- 172.21.15.146:0/3520840810 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25080b2210 0x7f25080c66f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 >> 172.21.15.146:0/3520840810 conn(0x7f25080682c0 msgr2=0x7f25080ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 shutdown_connections 2024-08-15T17:02:58.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:58.938+0000 7f250f78e700 1 -- 172.21.15.146:0/3520840810 wait complete. 2024-08-15T17:02:58.960 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-15T17:02:59.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.054+0000 7f90b3f65700 1 Processor -- start 2024-08-15T17:02:59.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.054+0000 7f90b3f65700 1 -- start start 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90ac0cc2e0 0x7f90ac0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cea40 0x7f90ac0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac0cf3e0 con 0x7f90ac159810 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ac0c5c60 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0ce120 con 0x7f90ac0cea40 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f90ac0cc2e0 0x7f90ac0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1d01700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f90ac159810 0x7f90ac0d0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41834/0 (socket says 172.21.15.146:41834) 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1d01700 1 -- 172.21.15.146:0/3330636318 learned_addr learned my addr 172.21.15.146:0/3330636318 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:02:59.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b2502700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cea40 0x7f90ac0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 -- 172.21.15.146:0/3330636318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cea40 msgr2=0x7f90ac0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cea40 0x7f90ac0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 -- 172.21.15.146:0/3330636318 >> v1:172.21.15.146:6789/0 conn(0x7f90ac159810 legacy=0x7f90ac0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b0cff700 1 -- 172.21.15.146:0/3330636318 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452313288 0 0) 0x7f90a8001110 con 0x7f90ac159810 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 -- 172.21.15.146:0/3330636318 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0ce390 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.071 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b1500700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac0cc2e0 0x7f90ac0b3270 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f909c0098e0 tx=0x7f909c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98776f563081da71 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b0cff700 1 -- 172.21.15.146:0/3330636318 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00b640 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b0cff700 1 -- 172.21.15.146:0/3330636318 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c00b7a0 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b0cff700 1 -- 172.21.15.146:0/3330636318 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00bb40 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- 172.21.15.146:0/3330636318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac0cc2e0 msgr2=0x7f90ac0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac0cc2e0 0x7f90ac0b3270 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f909c0098e0 tx=0x7f909c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- 172.21.15.146:0/3330636318 shutdown_connections 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cea40 0x7f90ac0ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 --2- 172.21.15.146:0/3330636318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac0cc2e0 0x7f90ac0b3270 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- 172.21.15.146:0/3330636318 >> 172.21.15.146:0/3330636318 conn(0x7f90ac067590 msgr2=0x7f90ac0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- 172.21.15.146:0/3330636318 shutdown_connections 2024-08-15T17:02:59.072 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.058+0000 7f90b3f65700 1 -- 172.21.15.146:0/3330636318 wait complete. 2024-08-15T17:02:59.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 Processor -- start 2024-08-15T17:02:59.073 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 -- start start 2024-08-15T17:02:59.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cc2e0 0x7f90ac14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.075 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac0cea40 0x7f90ac09da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac09a6a0 con 0x7f90ac159810 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac0c7690 con 0x7f90ac0cea40 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b3f65700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac09a8b0 con 0x7f90ac0cc2e0 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33454/0 (socket says 172.21.15.146:33454) 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 -- 172.21.15.146:0/1381341405 learned_addr learned my addr 172.21.15.146:0/1381341405 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b1d01700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cc2e0 0x7f90ac14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b1500700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac0cea40 0x7f90ac09da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 -- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cc2e0 msgr2=0x7f90ac14dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cc2e0 0x7f90ac14dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.076 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 -- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac0cea40 msgr2=0x7f90ac09da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac0cea40 0x7f90ac09da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 -- 172.21.15.146:0/1381341405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009420 con 0x7f90ac159810 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.062+0000 7f90b2502700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009e90 tx=0x7f90a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400caa0 con 0x7f90ac159810 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a400f040 con 0x7f90ac159810 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac09ab90 con 0x7f90ac159810 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400ce50 con 0x7f90ac159810 2024-08-15T17:02:59.077 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac09b150 con 0x7f90ac159810 2024-08-15T17:02:59.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac0c7820 con 0x7f90ac159810 2024-08-15T17:02:59.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f90a4014690 con 0x7f90ac159810 2024-08-15T17:02:59.078 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 0x7f9090062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:02:59.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f90a407d950 con 0x7f90ac159810 2024-08-15T17:02:59.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.066+0000 7f90b1d01700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 0x7f9090062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:02:59.079 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.070+0000 7f90b1d01700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 0x7f9090062d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007160 tx=0x7f90a8007f50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:02:59.086 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.074+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f90a4051b50 con 0x7f90ac159810 2024-08-15T17:02:59.310 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.298+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f90ac0645c0 con 0x7f90ac159810 2024-08-15T17:02:59.613 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.598+0000 7f90a1ffb700 1 -- 172.21.15.146:0/1381341405 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v147) v1 ==== 135+0+0 (secure 0 0 0) 0x7f90a40180c0 con 0x7f90ac159810 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 msgr2=0x7f9090062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 0x7f9090062d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007160 tx=0x7f90a8007f50 comp rx=0 tx=0).stop 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 msgr2=0x7f90ac09a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009e90 tx=0x7f90a400b040 comp rx=0 tx=0).stop 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 shutdown_connections 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9090060890 0x7f9090062d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f90ac159810 0x7f90ac09a160 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f90ac0cea40 0x7f90ac09da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.615 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 --2- 172.21.15.146:0/1381341405 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f90ac0cc2e0 0x7f90ac14dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:02:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 >> 172.21.15.146:0/1381341405 conn(0x7f90ac067590 msgr2=0x7f90ac157ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:02:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 shutdown_connections 2024-08-15T17:02:59.616 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:02:59.602+0000 7f90b3f65700 1 -- 172.21.15.146:0/1381341405 wait complete. 2024-08-15T17:02:59.636 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T17:02:59.637 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi146 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000940580 --yes-i-really-mean-it' 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner: 2024-08-15T17:02:59.638 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:Ran 1 test in 166.464s 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-15T17:02:59.639 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi146 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000940580 --yes-i-really-mean-it' 2024-08-15T17:02:59.640 INFO:tasks.cephfs_test_runner: 2024-08-15T17:02:59.641 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:02:59.868 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f3e9e0b963554613927d2255e4156aa1 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2c991c196a651f8ef0f36c7763ec87a241b25549/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-15T17:02:59.870 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-08-15T17:02:59.883 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-15T17:02:59.929 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-15T17:02:59.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:59.930 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-08-15T17:02:59.943 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-15T17:02:59.943 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-15T17:02:59.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:59.943 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-15T17:02:59.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:02:59.961 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-15T17:03:00.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:03:00.042 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-08-15T17:03:00.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:03:00.208 DEBUG:teuthology.orchestra.run.smithi146:> ip netns list 2024-08-15T17:03:00.216 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-15T17:03:00.216 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-15T17:03:00.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:03:00.216 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-15T17:03:00.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:03:00.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-15T17:03:00.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-15T17:03:00.288 DEBUG:teuthology.orchestra.run.smithi146:> sudo ip link delete ceph-brx 2024-08-15T17:03:00.434 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-15T17:03:00.444 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-15T17:03:00.444 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:03:00.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 Processor -- start 2024-08-15T17:03:00.554 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 -- start start 2024-08-15T17:03:00.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb0640af340 0x7fb064067840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640cc8c0 con 0x7fb064067d80 2024-08-15T17:03:00.555 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0640cbca0 con 0x7fb064156d80 2024-08-15T17:03:00.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.542+0000 7fb06aa8f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb064068200 con 0x7fb0640af340 2024-08-15T17:03:00.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06a28e700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb064067d80 0x7fb064157150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41840/0 (socket says 172.21.15.146:41840) 2024-08-15T17:03:00.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06a28e700 1 -- 172.21.15.146:0/1541399166 learned_addr learned my addr 172.21.15.146:0/1541399166 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:00.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06928c700 1 --2- 172.21.15.146:0/1541399166 >> v2:172.21.15.161:3300/0 conn(0x7fb0640af340 0x7fb064067840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06928c700 1 -- 172.21.15.146:0/1541399166 >> v1:172.21.15.161:6789/0 conn(0x7fb064156d80 legacy=0x7fb0640aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06928c700 1 -- 172.21.15.146:0/1541399166 >> v1:172.21.15.146:6789/0 conn(0x7fb064067d80 legacy=0x7fb064157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06928c700 1 -- 172.21.15.146:0/1541399166 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064068450 con 0x7fb0640af340 2024-08-15T17:03:00.557 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb06928c700 1 --2- 172.21.15.146:0/1541399166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb064067840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0540098e0 tx=0x7fb0540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=114770a80d4cdbf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb068a8b700 1 -- 172.21.15.146:0/1541399166 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054009e80 con 0x7fb0640af340 2024-08-15T17:03:00.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb068a8b700 1 -- 172.21.15.146:0/1541399166 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05400b760 con 0x7fb0640af340 2024-08-15T17:03:00.558 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.546+0000 7fb068a8b700 1 -- 172.21.15.146:0/1541399166 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05400bae0 con 0x7fb0640af340 2024-08-15T17:03:00.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 -- 172.21.15.146:0/1541399166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 msgr2=0x7fb064067840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/1541399166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb064067840 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb0540098e0 tx=0x7fb0540091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 -- 172.21.15.146:0/1541399166 shutdown_connections 2024-08-15T17:03:00.560 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/1541399166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb064067840 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 -- 172.21.15.146:0/1541399166 >> 172.21.15.146:0/1541399166 conn(0x7fb06406e070 msgr2=0x7fb0640b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:00.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 -- 172.21.15.146:0/1541399166 shutdown_connections 2024-08-15T17:03:00.561 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 -- 172.21.15.146:0/1541399166 wait complete. 2024-08-15T17:03:00.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.550+0000 7fb06aa8f700 1 Processor -- start 2024-08-15T17:03:00.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- start start 2024-08-15T17:03:00.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 0x7fb064151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb06414da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0641500a0 con 0x7fb064156d80 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06414f490 con 0x7fb0640af340 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06414f770 con 0x7fb064067d80 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33458/0 (socket says 172.21.15.146:33458) 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 -- 172.21.15.146:0/4129511359 learned_addr learned my addr 172.21.15.146:0/4129511359 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06928c700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb06414da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb069a8d700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 0x7fb064151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 -- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 msgr2=0x7fb064151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 0x7fb064151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 -- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 msgr2=0x7fb06414da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb06414da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 -- 172.21.15.146:0/4129511359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb054009420 con 0x7fb064156d80 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb069a8d700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 0x7fb064151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06a28e700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00cd50 tx=0x7fb05c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c00ab70 con 0x7fb064156d80 2024-08-15T17:03:00.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c00acd0 con 0x7fb064156d80 2024-08-15T17:03:00.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06414fa50 con 0x7fb064156d80 2024-08-15T17:03:00.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c007340 con 0x7fb064156d80 2024-08-15T17:03:00.570 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.554+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0641515d0 con 0x7fb064156d80 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064150210 con 0x7fb064156d80 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb05c0076f0 con 0x7fb064156d80 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb059ffb700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 0x7fb048062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fb05c07cd80 con 0x7fb064156d80 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb069a8d700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 0x7fb048062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.571 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.558+0000 7fb069a8d700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 0x7fb048062d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb060007fd0 tx=0x7fb060009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.562+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fb05c04d4f0 con 0x7fb064156d80 2024-08-15T17:03:00.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.730+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0640638f0 con 0x7fb064156d80 2024-08-15T17:03:00.744 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb059ffb700 1 -- 172.21.15.146:0/4129511359 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12443 (secure 0 0 0) 0x7fb05c050b10 con 0x7fb064156d80 2024-08-15T17:03:00.744 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:00.744 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:02:58.585353+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.146:6835/163298469":"2024-08-16T17:02:55.567258+0000","172.21.15.146:6834/163298469":"2024-08-16T17:02:55.567258+0000","172.21.15.161:6837/2559773181":"2024-08-16T17:02:49.326606+0000","172.21.15.161:6836/2559773181":"2024-08-16T17:02:49.326606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:03:00.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 msgr2=0x7fb048062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 0x7fb048062d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb060007fd0 tx=0x7fb060009040 comp rx=0 tx=0).stop 2024-08-15T17:03:00.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 msgr2=0x7fb064150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.746 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00cd50 tx=0x7fb05c001ab0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 shutdown_connections 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb048060890 0x7fb048062d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb064156d80 0x7fb064150530 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb0640af340 0x7fb06414da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 --2- 172.21.15.146:0/4129511359 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb064067d80 0x7fb064151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.747 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 >> 172.21.15.146:0/4129511359 conn(0x7fb06406e070 msgr2=0x7fb0640afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:00.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 shutdown_connections 2024-08-15T17:03:00.748 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.734+0000 7fb06aa8f700 1 -- 172.21.15.146:0/4129511359 wait complete. 2024-08-15T17:03:00.757 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-15T17:03:00.757 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:03:00.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.850+0000 7f5906886700 1 Processor -- start 2024-08-15T17:03:00.863 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.850+0000 7f5906886700 1 -- start start 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5900068840 0x7f5900068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f590006ac00 0x7f590006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000698b0 con 0x7f59000ca000 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59000aef40 con 0x7f5900068840 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590006b1a0 con 0x7f590006ac00 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5900068840 0x7f5900068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f5900068840 0x7f5900068c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33466/0 (socket says 172.21.15.146:33466) 2024-08-15T17:03:00.865 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 -- 172.21.15.146:0/3893162389 learned_addr learned my addr 172.21.15.146:0/3893162389 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905083700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f590006ac00 0x7f590006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 -- 172.21.15.146:0/3893162389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f590006ac00 msgr2=0x7f590006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f590006ac00 0x7f590006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 -- 172.21.15.146:0/3893162389 >> v1:172.21.15.161:6789/0 conn(0x7f59000ca000 legacy=0x7f5900069d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 -- 172.21.15.146:0/3893162389 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000ae300 con 0x7f5900068840 2024-08-15T17:03:00.866 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5905884700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900068840 0x7f5900068c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0099b0 tx=0x7f58fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68bb56e423f981 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.867 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5904882700 1 -- 172.21.15.146:0/3893162389 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc009ea0 con 0x7f5900068840 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5904882700 1 -- 172.21.15.146:0/3893162389 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc00b860 con 0x7f5900068840 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5904882700 1 -- 172.21.15.146:0/3893162389 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc00bc00 con 0x7f5900068840 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- 172.21.15.146:0/3893162389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900068840 msgr2=0x7f5900068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900068840 0x7f5900068c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0099b0 tx=0x7f58fc009250 comp rx=0 tx=0).stop 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- 172.21.15.146:0/3893162389 shutdown_connections 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f590006ac00 0x7f590006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 --2- 172.21.15.146:0/3893162389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5900068840 0x7f5900068c10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.868 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- 172.21.15.146:0/3893162389 >> 172.21.15.146:0/3893162389 conn(0x7f590006d800 msgr2=0x7f590006dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:00.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- 172.21.15.146:0/3893162389 shutdown_connections 2024-08-15T17:03:00.869 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.854+0000 7f5906886700 1 -- 172.21.15.146:0/3893162389 wait complete. 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 Processor -- start 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- start start 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 0x7f5900156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 0x7f59000ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59001553c0 con 0x7f590006ac00 2024-08-15T17:03:00.871 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59001556a0 con 0x7f5900068840 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900155980 con 0x7f59000ca000 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33472/0 (socket says 172.21.15.146:33472) 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 -- 172.21.15.146:0/447203008 learned_addr learned my addr 172.21.15.146:0/447203008 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905884700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 0x7f5900156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906085700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 0x7f59000ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 -- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 msgr2=0x7f59000ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 0x7f59000ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.872 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 -- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 msgr2=0x7f5900156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 0x7f5900156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 -- 172.21.15.146:0/447203008 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0094f0 con 0x7f590006ac00 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905884700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 0x7f5900156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906085700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 0x7f59000ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5905083700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ec80 tx=0x7f58f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000c9d0 con 0x7f590006ac00 2024-08-15T17:03:00.873 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000cb30 con 0x7f590006ac00 2024-08-15T17:03:00.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5900155c60 con 0x7f590006ac00 2024-08-15T17:03:00.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0013470 con 0x7f590006ac00 2024-08-15T17:03:00.874 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.858+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5900155090 con 0x7f590006ac00 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59000638f0 con 0x7f590006ac00 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f58f0013820 con 0x7f590006ac00 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f58f5ffb700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 0x7f58e4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f58f007cc80 con 0x7f590006ac00 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f5905884700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 0x7f58e4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:00.875 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.862+0000 7f5905884700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 0x7f58e4062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0099b0 tx=0x7f58fc017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:00.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:00.870+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f58f0083080 con 0x7f590006ac00 2024-08-15T17:03:01.028 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.018+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5900067350 con 0x7f58e4060860 2024-08-15T17:03:01.033 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.022+0000 7f58f5ffb700 1 -- 172.21.15.146:0/447203008 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48741 (secure 0 0 0) 0x7f5900067350 con 0x7f58e4060860 2024-08-15T17:03:01.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:01.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 msgr2=0x7f58e4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 0x7f58e4062d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0099b0 tx=0x7f58fc017000 comp rx=0 tx=0).stop 2024-08-15T17:03:01.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 msgr2=0x7f5900145e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ec80 tx=0x7f58f00091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 shutdown_connections 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f58e4060860 0x7f58e4062d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59000ca000 0x7f59000ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f590006ac00 0x7f5900145e50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 --2- 172.21.15.146:0/447203008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5900068840 0x7f5900156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 >> 172.21.15.146:0/447203008 conn(0x7f590006d800 msgr2=0x7f590015b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 shutdown_connections 2024-08-15T17:03:01.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.026+0000 7f5906886700 1 -- 172.21.15.146:0/447203008 wait complete. 2024-08-15T17:03:01.040 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:03:01.053 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-08-15T17:02:59.721461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2396204,"kb_used_data":4848,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578516,"statfs":{"total":773094113280,"available":770640400384,"internally_reserved":0,"allocated":4964352,"data_stored":2734436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001792"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T17:02:51.355802+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:02:51.355802+0000","last_peered":"2024-08-15T17:02:51.355802+0000","last_clean":"2024-08-15T17:02:51.355802+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:02:51.355802+0000","last_undegraded":"2024-08-15T17:02:51.355802+0000","last_fullsized":"2024-08-15T17:02:51.355802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1060864,"data_stored":783431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86699999999999999,"15min":0.83199999999999996},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.2430000000000001,"5min":1.258,"15min":1.258},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.335,"15min":1.335},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.169,"5min":1.415,"15min":1.415},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1479999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.024}]},{"osd":2,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.69099999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2230000000000001,"5min":1.329,"15min":1.329},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.105},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.518,"15min":1.518},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97699999999999998,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.264,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.997},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1,"15min":0.98999999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2430000000000001,"5min":1.292,"15min":1.292},"last":0.752}]},{"osd":4,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.63800000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.327,"5min":1.357,"15min":1.357},"last":1.179},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.048,"15min":1.056},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.304,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.099}]},{"osd":5,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1.0700000000000001},"min":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.26,"5min":1.409,"15min":1.409},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.125}]},{"osd":6,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0649999999999999,"15min":1.079},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2210000000000001,"5min":1.387,"15min":1.387},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.93100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.381,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.407}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":471040,"data_stored":193633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.90400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.748},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95499999999999996,"15min":0.96099999999999997},"min":{"1min":0.77200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":2.097,"15min":2.097},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.72299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.121,"5min":1.25,"15min":1.25},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.61099999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1180000000000001,"5min":1.444,"15min":1.444},"last":0.81299999999999994}]},{"osd":2,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.379,"15min":1.379},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.121,"5min":1.373,"15min":1.373},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3580000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.65900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3420000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.397,"5min":1.698,"15min":1.698},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0900000000000001,"15min":1.117},"min":{"1min":0.745,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.46,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.151,"5min":1.216,"15min":1.244},"min":{"1min":0.90000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.603,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":0.77800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301976,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069864,"statfs":{"total":96636764160,"available":96327540736,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.876,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.077}]},{"osd":1,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1140000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.163}]},{"osd":2,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.286,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.103,"15min":1.129},"min":{"1min":0.879,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.389,"5min":1.72,"15min":1.72},"last":1.3149999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.02,"15min":1.042},"min":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2290000000000001,"5min":1.417,"15min":1.417},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0940000000000001,"15min":1.1240000000000001},"min":{"1min":0.78400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2969999999999999,"5min":1.593,"15min":1.593},"last":1.1120000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.028,"15min":1.075},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1659999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.161,"15min":1.1850000000000001},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3560000000000001,"5min":1.889,"15min":1.889},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.133,"15min":1.1739999999999999},"min":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3420000000000001,"5min":1.734,"15min":1.734},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.1839999999999999,"15min":1.23},"min":{"1min":0.86599999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.268,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.3},"min":{"1min":1.004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4390000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.444},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3180000000000001,"15min":1.3600000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.4590000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.5389999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301992,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069848,"statfs":{"total":96636764160,"available":96327524352,"internally_reserved":0,"allocated":487424,"data_stored":200726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.875},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2450000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.626,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.747}]},{"osd":1,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.875},{"interface":"front","average":{"1min":1.048,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.749,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.94299999999999995},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.23,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.008},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93000000000000005,"15min":0.92200000000000004},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.93700000000000006,"15min":0.94299999999999995},"min":{"1min":0.71099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.181,"5min":1.794,"15min":1.794},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.048,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0249999999999999,"15min":1.075},"min":{"1min":0.76100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.0609999999999999,"15min":1.095},"min":{"1min":0.79900000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.048}]},{"osd":6,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.075,"15min":1.101},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.333,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.504,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.111,"15min":1.135},"min":{"1min":0.90700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2470000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":1.038,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.587,"5min":1.99,"15min":1.99},"last":1.3380000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":475136,"data_stored":193599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.843,"15min":1.843},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.55700000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92400000000000004,"5min":1.202,"15min":1.202},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0129999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.65600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0289999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0129999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.997},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1859999999999999,"5min":1.956,"15min":1.956},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.071,"15min":1.083},"min":{"1min":0.92500000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.151,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.95199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.81799999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.218,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.077,"15min":1.101},"min":{"1min":0.69399999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.196,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0920000000000001,"15min":1.107},"min":{"1min":0.81299999999999994,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2390000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.86499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.1679999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.002}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.90900000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.222,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0920000000000001,"15min":1.0800000000000001},"min":{"1min":0.93000000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.337,"5min":1.573,"15min":1.573},"last":1.0669999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":479232,"data_stored":194547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.008,"15min":1.008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.54000000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.66500000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.70799999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.82099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.26,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.125},{"interface":"front","average":{"1min":1.054,"5min":1.022,"15min":1.012},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.448,"5min":1.734,"15min":1.734},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0189999999999999,"15min":0.997},"min":{"1min":0.72699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.008,"15min":1.018},"min":{"1min":0.79200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.173,"5min":1.234,"15min":1.234},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.339,"5min":1.375,"15min":1.375},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3089999999999999,"5min":1.397,"15min":1.397},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1180000000000001,"15min":1.1040000000000001},"min":{"1min":0.91200000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.98999999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.268,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.83499999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.123},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.22,"15min":1.22},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0960000000000001,"5min":1.167,"15min":1.167},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.69599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.004,"15min":1.018},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4890000000000001,"5min":1.647,"15min":1.647},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.76600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4510000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1000000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.248,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.073},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.071},"min":{"1min":0.94599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.075,"15min":1.095},"min":{"1min":0.81100000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.331,"5min":1.96,"15min":1.96},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.3580000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.2529999999999999},"min":{"1min":0.90600000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.4510000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.274,"5min":1.153,"15min":1.125},"min":{"1min":1.0469999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.123}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1056768,"data_stored":782414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.119,"15min":1.119},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0169999999999999,"5min":1.179,"15min":1.179},"last":0.495}]},{"osd":2,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.78700000000000003},"min":{"1min":0.752,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0660000000000001,"5min":1.385,"15min":1.385},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.044,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.95899999999999996,"15min":0.94899999999999995},"min":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.252,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.304,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.377,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.071,"15min":1.0569999999999999},"min":{"1min":0.92200000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.327,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.044,"15min":1.038},"min":{"1min":0.84899999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.169,"15min":1.1679999999999999},"min":{"1min":1.0489999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.385,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.125},"min":{"1min":0.96299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.335,"5min":1.877,"15min":1.877},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":1.052,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4239999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:03:01.056 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:03:01.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.146+0000 7fe7425fe700 1 Processor -- start 2024-08-15T17:03:01.159 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.146+0000 7fe7425fe700 1 -- start start 2024-08-15T17:03:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe73c0d0010 0x7fe73c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0cf190 con 0x7fe73c159040 2024-08-15T17:03:01.161 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0d09a0 con 0x7fe73c0d0010 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0ce5e0 con 0x7fe73c15cea0 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe73c0d0010 0x7fe73c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fe73c0d0010 0x7fe73c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33474/0 (socket says 172.21.15.146:33474) 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 -- 172.21.15.146:0/632310238 learned_addr learned my addr 172.21.15.146:0/632310238 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7415fc700 1 --1- 172.21.15.146:0/632310238 >> v1:172.21.15.146:6789/0 conn(0x7fe73c159040 0x7fe73c0cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41852/0 (socket says 172.21.15.146:41852) 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe741dfd700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.162 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 -- 172.21.15.146:0/632310238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 msgr2=0x7fe73c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:01.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 -- 172.21.15.146:0/632310238 >> v1:172.21.15.146:6789/0 conn(0x7fe73c159040 legacy=0x7fe73c0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 -- 172.21.15.146:0/632310238 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0ce860 con 0x7fe73c0d0010 2024-08-15T17:03:01.163 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe741dfd700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:01.166 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe740dfb700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c0d0010 0x7fe73c0d0460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0099b0 tx=0x7fe72c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e5b40bdf888795 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe733fff700 1 -- 172.21.15.146:0/632310238 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00b860 con 0x7fe73c0d0010 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe733fff700 1 -- 172.21.15.146:0/632310238 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c00b9c0 con 0x7fe73c0d0010 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe733fff700 1 -- 172.21.15.146:0/632310238 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00bd80 con 0x7fe73c0d0010 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- 172.21.15.146:0/632310238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c0d0010 msgr2=0x7fe73c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c0d0010 0x7fe73c0d0460 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0099b0 tx=0x7fe72c009250 comp rx=0 tx=0).stop 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- 172.21.15.146:0/632310238 shutdown_connections 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.167 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 --2- 172.21.15.146:0/632310238 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c0d0010 0x7fe73c0d0460 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- 172.21.15.146:0/632310238 >> 172.21.15.146:0/632310238 conn(0x7fe73c068250 msgr2=0x7fe73c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- 172.21.15.146:0/632310238 shutdown_connections 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.150+0000 7fe7425fe700 1 -- 172.21.15.146:0/632310238 wait complete. 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 Processor -- start 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- start start 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 0x7fe73c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.168 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0ccd40 con 0x7fe73c159040 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0b0090 con 0x7fe73c0d0010 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0b0370 con 0x7fe73c15cea0 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33482/0 (socket says 172.21.15.146:33482) 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 -- 172.21.15.146:0/763651252 learned_addr learned my addr 172.21.15.146:0/763651252 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7415fc700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 0x7fe73c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.169 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe741dfd700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 -- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 msgr2=0x7fe73c15c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 -- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 msgr2=0x7fe73c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 0x7fe73c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 -- 172.21.15.146:0/763651252 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0094f0 con 0x7fe73c159040 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7415fc700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 0x7fe73c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe741dfd700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-15T17:03:01.170 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe740dfb700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0090b0 tx=0x7fe72c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c026450 con 0x7fe73c159040 2024-08-15T17:03:01.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c0265f0 con 0x7fe73c159040 2024-08-15T17:03:01.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c0b0650 con 0x7fe73c159040 2024-08-15T17:03:01.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0269b0 con 0x7fe73c159040 2024-08-15T17:03:01.171 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.154+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c16a030 con 0x7fe73c159040 2024-08-15T17:03:01.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe72c01a420 con 0x7fe73c159040 2024-08-15T17:03:01.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0b0b50 con 0x7fe73c159040 2024-08-15T17:03:01.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7317fa700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 0x7fe720062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe72c07e390 con 0x7fe73c159040 2024-08-15T17:03:01.172 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7415fc700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 0x7fe720062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.173 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.158+0000 7fe7415fc700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 0x7fe720062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7380017a0 tx=0x7fe738009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.179 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.166+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe72c04eb00 con 0x7fe73c159040 2024-08-15T17:03:01.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.314+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe73c0643a0 con 0x7fe720060860 2024-08-15T17:03:01.332 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7317fa700 1 -- 172.21.15.146:0/763651252 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48741 (secure 0 0 0) 0x7fe73c0643a0 con 0x7fe720060860 2024-08-15T17:03:01.333 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:01.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 msgr2=0x7fe720062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 0x7fe720062d10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe7380017a0 tx=0x7fe738009040 comp rx=0 tx=0).stop 2024-08-15T17:03:01.335 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 msgr2=0x7fe73c15c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0090b0 tx=0x7fe72c017000 comp rx=0 tx=0).stop 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 shutdown_connections 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe720060860 0x7fe720062d10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe73c15cea0 0x7fe73c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe73c159040 0x7fe73c15c300 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 --2- 172.21.15.146:0/763651252 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe73c0d0010 0x7fe73c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 >> 172.21.15.146:0/763651252 conn(0x7fe73c068250 msgr2=0x7fe73c157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.336 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 shutdown_connections 2024-08-15T17:03:01.337 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.322+0000 7fe7425fe700 1 -- 172.21.15.146:0/763651252 wait complete. 2024-08-15T17:03:01.337 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:03:01.351 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-08-15T17:02:59.721461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2396204,"kb_used_data":4848,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578516,"statfs":{"total":773094113280,"available":770640400384,"internally_reserved":0,"allocated":4964352,"data_stored":2734436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001792"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T17:02:51.355802+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:02:51.355802+0000","last_peered":"2024-08-15T17:02:51.355802+0000","last_clean":"2024-08-15T17:02:51.355802+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:02:51.355802+0000","last_undegraded":"2024-08-15T17:02:51.355802+0000","last_fullsized":"2024-08-15T17:02:51.355802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1060864,"data_stored":783431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86699999999999999,"15min":0.83199999999999996},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.2430000000000001,"5min":1.258,"15min":1.258},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.335,"15min":1.335},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.169,"5min":1.415,"15min":1.415},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1479999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.024}]},{"osd":2,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.69099999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2230000000000001,"5min":1.329,"15min":1.329},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.105},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.518,"15min":1.518},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97699999999999998,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.264,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.997},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1,"15min":0.98999999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2430000000000001,"5min":1.292,"15min":1.292},"last":0.752}]},{"osd":4,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.63800000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.327,"5min":1.357,"15min":1.357},"last":1.179},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.048,"15min":1.056},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.304,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.099}]},{"osd":5,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1.0700000000000001},"min":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.26,"5min":1.409,"15min":1.409},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.125}]},{"osd":6,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0649999999999999,"15min":1.079},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2210000000000001,"5min":1.387,"15min":1.387},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.93100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.381,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.407}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":471040,"data_stored":193633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.90400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.748},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95499999999999996,"15min":0.96099999999999997},"min":{"1min":0.77200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":2.097,"15min":2.097},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.72299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.121,"5min":1.25,"15min":1.25},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.61099999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1180000000000001,"5min":1.444,"15min":1.444},"last":0.81299999999999994}]},{"osd":2,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.379,"15min":1.379},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.121,"5min":1.373,"15min":1.373},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3580000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.65900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3420000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.397,"5min":1.698,"15min":1.698},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0900000000000001,"15min":1.117},"min":{"1min":0.745,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.46,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.151,"5min":1.216,"15min":1.244},"min":{"1min":0.90000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.603,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":0.77800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301976,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069864,"statfs":{"total":96636764160,"available":96327540736,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.876,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.077}]},{"osd":1,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1140000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.163}]},{"osd":2,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.286,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.103,"15min":1.129},"min":{"1min":0.879,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.389,"5min":1.72,"15min":1.72},"last":1.3149999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.02,"15min":1.042},"min":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2290000000000001,"5min":1.417,"15min":1.417},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0940000000000001,"15min":1.1240000000000001},"min":{"1min":0.78400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2969999999999999,"5min":1.593,"15min":1.593},"last":1.1120000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.028,"15min":1.075},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1659999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.161,"15min":1.1850000000000001},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3560000000000001,"5min":1.889,"15min":1.889},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.133,"15min":1.1739999999999999},"min":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3420000000000001,"5min":1.734,"15min":1.734},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.1839999999999999,"15min":1.23},"min":{"1min":0.86599999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.268,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.3},"min":{"1min":1.004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4390000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.444},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3180000000000001,"15min":1.3600000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.4590000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.5389999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301992,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069848,"statfs":{"total":96636764160,"available":96327524352,"internally_reserved":0,"allocated":487424,"data_stored":200726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.875},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2450000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.626,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.747}]},{"osd":1,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.875},{"interface":"front","average":{"1min":1.048,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.749,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.94299999999999995},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.23,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.008},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93000000000000005,"15min":0.92200000000000004},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.93700000000000006,"15min":0.94299999999999995},"min":{"1min":0.71099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.181,"5min":1.794,"15min":1.794},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.048,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0249999999999999,"15min":1.075},"min":{"1min":0.76100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.0609999999999999,"15min":1.095},"min":{"1min":0.79900000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.048}]},{"osd":6,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.075,"15min":1.101},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.333,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.504,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.111,"15min":1.135},"min":{"1min":0.90700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2470000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":1.038,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.587,"5min":1.99,"15min":1.99},"last":1.3380000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":475136,"data_stored":193599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.843,"15min":1.843},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.55700000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92400000000000004,"5min":1.202,"15min":1.202},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0129999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.65600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0289999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0129999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.997},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1859999999999999,"5min":1.956,"15min":1.956},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.071,"15min":1.083},"min":{"1min":0.92500000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.151,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.95199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.81799999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.218,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.077,"15min":1.101},"min":{"1min":0.69399999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.196,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0920000000000001,"15min":1.107},"min":{"1min":0.81299999999999994,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2390000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.86499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.1679999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.002}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.90900000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.222,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0920000000000001,"15min":1.0800000000000001},"min":{"1min":0.93000000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.337,"5min":1.573,"15min":1.573},"last":1.0669999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":479232,"data_stored":194547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.008,"15min":1.008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.54000000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.66500000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.70799999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.82099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.26,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.125},{"interface":"front","average":{"1min":1.054,"5min":1.022,"15min":1.012},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.448,"5min":1.734,"15min":1.734},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0189999999999999,"15min":0.997},"min":{"1min":0.72699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.008,"15min":1.018},"min":{"1min":0.79200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.173,"5min":1.234,"15min":1.234},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.339,"5min":1.375,"15min":1.375},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3089999999999999,"5min":1.397,"15min":1.397},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1180000000000001,"15min":1.1040000000000001},"min":{"1min":0.91200000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.98999999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.268,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.83499999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.123},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.22,"15min":1.22},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0960000000000001,"5min":1.167,"15min":1.167},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.69599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.004,"15min":1.018},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4890000000000001,"5min":1.647,"15min":1.647},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.76600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4510000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1000000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.248,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.073},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.071},"min":{"1min":0.94599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.075,"15min":1.095},"min":{"1min":0.81100000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.331,"5min":1.96,"15min":1.96},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.3580000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.2529999999999999},"min":{"1min":0.90600000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.4510000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.274,"5min":1.153,"15min":1.125},"min":{"1min":1.0469999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.123}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1056768,"data_stored":782414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.119,"15min":1.119},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0169999999999999,"5min":1.179,"15min":1.179},"last":0.495}]},{"osd":2,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.78700000000000003},"min":{"1min":0.752,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0660000000000001,"5min":1.385,"15min":1.385},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.044,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.95899999999999996,"15min":0.94899999999999995},"min":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.252,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.304,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.377,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.071,"15min":1.0569999999999999},"min":{"1min":0.92200000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.327,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.044,"15min":1.038},"min":{"1min":0.84899999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.169,"15min":1.1679999999999999},"min":{"1min":1.0489999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.385,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.125},"min":{"1min":0.96299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.335,"5min":1.877,"15min":1.877},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":1.052,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4239999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:03:01.353 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-15T17:03:01.353 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:03:01.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 Processor -- start 2024-08-15T17:03:01.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 -- start start 2024-08-15T17:03:01.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76440af030 0x7f76440a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76440a9260 0x7f76440b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76440b0270 con 0x7f764415d540 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440af660 con 0x7f76440af030 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76494a1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440af940 con 0x7f76440a9260 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76440af030 0x7f76440a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f7642ffd700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f764415d540 0x7f764406e830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41856/0 (socket says 172.21.15.146:41856) 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f76440af030 0x7f76440a8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33490/0 (socket says 172.21.15.146:33490) 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f7642ffd700 1 -- 172.21.15.146:0/3077107332 learned_addr learned my addr 172.21.15.146:0/3077107332 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:01.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76437fe700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76440a9260 0x7f76440b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 -- 172.21.15.146:0/3077107332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76440a9260 msgr2=0x7f76440b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:01.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76440a9260 0x7f76440b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 -- 172.21.15.146:0/3077107332 >> v1:172.21.15.146:6789/0 conn(0x7f764415d540 legacy=0x7f764406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 -- 172.21.15.146:0/3077107332 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440afc20 con 0x7f76440af030 2024-08-15T17:03:01.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.446+0000 7f76427fc700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440af030 0x7f76440a8d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f76340098e0 tx=0x7f76340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ea8ffd6b7caac8 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f7641ffb700 1 -- 172.21.15.146:0/3077107332 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400b640 con 0x7f76440af030 2024-08-15T17:03:01.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f7641ffb700 1 -- 172.21.15.146:0/3077107332 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763400b7a0 con 0x7f76440af030 2024-08-15T17:03:01.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f7641ffb700 1 -- 172.21.15.146:0/3077107332 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400bb40 con 0x7f76440af030 2024-08-15T17:03:01.462 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 -- 172.21.15.146:0/3077107332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440af030 msgr2=0x7f76440a8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440af030 0x7f76440a8d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f76340098e0 tx=0x7f76340091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 -- 172.21.15.146:0/3077107332 shutdown_connections 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76440a9260 0x7f76440b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 --2- 172.21.15.146:0/3077107332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440af030 0x7f76440a8d20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 -- 172.21.15.146:0/3077107332 >> 172.21.15.146:0/3077107332 conn(0x7f7644066e00 msgr2=0x7f7644067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 -- 172.21.15.146:0/3077107332 shutdown_connections 2024-08-15T17:03:01.463 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.450+0000 7f76494a1700 1 -- 172.21.15.146:0/3077107332 wait complete. 2024-08-15T17:03:01.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 Processor -- start 2024-08-15T17:03:01.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- start start 2024-08-15T17:03:01.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76440af030 0x7f76440981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 0x7f76440986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.467 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440ca1a0 con 0x7f76440a9260 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644098c30 con 0x7f76440af030 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7644097480 con 0x7f764415d540 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f7642ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f7642ffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33502/0 (socket says 172.21.15.146:33502) 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f7642ffd700 1 -- 172.21.15.146:0/3678425899 learned_addr learned my addr 172.21.15.146:0/3678425899 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 0x7f76440986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76427fc700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76440af030 0x7f76440981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 -- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76440af030 msgr2=0x7f76440981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76440af030 0x7f76440981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 -- 172.21.15.146:0/3678425899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 msgr2=0x7f76440ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 -- 172.21.15.146:0/3678425899 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7634009420 con 0x7f764415d540 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f7642ffd700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76437fe700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 0x7f76440986f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f763c009e50 tx=0x7f763c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00caa0 con 0x7f764415d540 2024-08-15T17:03:01.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00f040 con 0x7f764415d540 2024-08-15T17:03:01.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7644098e30 con 0x7f764415d540 2024-08-15T17:03:01.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00ce70 con 0x7f764415d540 2024-08-15T17:03:01.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.454+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7644097110 con 0x7f764415d540 2024-08-15T17:03:01.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7644097610 con 0x7f764415d540 2024-08-15T17:03:01.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f763c014690 con 0x7f764415d540 2024-08-15T17:03:01.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f763affd700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 0x7f7624062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f763c07d220 con 0x7f764415d540 2024-08-15T17:03:01.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f7642ffd700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 0x7f7624062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.472 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.458+0000 7f7642ffd700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 0x7f7624062d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f762c007fd0 tx=0x7f762c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.482 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.470+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f763c04d910 con 0x7f764415d540 2024-08-15T17:03:01.639 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.630+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76440645c0 con 0x7f7624060860 2024-08-15T17:03:01.645 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.634+0000 7f763affd700 1 -- 172.21.15.146:0/3678425899 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48741 (secure 0 0 0) 0x7f76440645c0 con 0x7f7624060860 2024-08-15T17:03:01.645 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:01.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.634+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 msgr2=0x7f7624062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.647 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.634+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 0x7f7624062d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f762c007fd0 tx=0x7f762c009040 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.634+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 msgr2=0x7f76440986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.634+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 0x7f76440986f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f763c009e50 tx=0x7f763c00b040 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 shutdown_connections 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f7624060860 0x7f7624062d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f764415d540 0x7f76440986f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76440af030 0x7f76440981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 --2- 172.21.15.146:0/3678425899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f76440a9260 0x7f76440ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.648 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 >> 172.21.15.146:0/3678425899 conn(0x7f7644066e00 msgr2=0x7f764406d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 shutdown_connections 2024-08-15T17:03:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.638+0000 7f76494a1700 1 -- 172.21.15.146:0/3678425899 wait complete. 2024-08-15T17:03:01.649 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:03:01.660 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-08-15T17:02:59.721461+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2396204,"kb_used_data":4848,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578516,"statfs":{"total":773094113280,"available":770640400384,"internally_reserved":0,"allocated":4964352,"data_stored":2734436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001792"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-08-15T17:02:51.355802+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:02:51.355802+0000","last_peered":"2024-08-15T17:02:51.355802+0000","last_clean":"2024-08-15T17:02:51.355802+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:02:51.355802+0000","last_undegraded":"2024-08-15T17:02:51.355802+0000","last_fullsized":"2024-08-15T17:02:51.355802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1060864,"data_stored":783431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86699999999999999,"15min":0.83199999999999996},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.2430000000000001,"5min":1.258,"15min":1.258},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.335,"15min":1.335},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.169,"5min":1.415,"15min":1.415},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1479999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.024}]},{"osd":2,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.69099999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2230000000000001,"5min":1.329,"15min":1.329},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.105},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.518,"15min":1.518},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97699999999999998,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.264,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.997},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1,"15min":0.98999999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2430000000000001,"5min":1.292,"15min":1.292},"last":0.752}]},{"osd":4,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.63800000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.327,"5min":1.357,"15min":1.357},"last":1.179},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.048,"15min":1.056},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.304,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.099}]},{"osd":5,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1.0700000000000001},"min":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.26,"5min":1.409,"15min":1.409},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.125}]},{"osd":6,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0649999999999999,"15min":1.079},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2210000000000001,"5min":1.387,"15min":1.387},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.93100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.381,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.407}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":471040,"data_stored":193633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.90400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.748},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95499999999999996,"15min":0.96099999999999997},"min":{"1min":0.77200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":2.097,"15min":2.097},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.72299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.121,"5min":1.25,"15min":1.25},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.61099999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1180000000000001,"5min":1.444,"15min":1.444},"last":0.81299999999999994}]},{"osd":2,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.379,"15min":1.379},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.121,"5min":1.373,"15min":1.373},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3580000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.65900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3420000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.397,"5min":1.698,"15min":1.698},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0900000000000001,"15min":1.117},"min":{"1min":0.745,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.46,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.151,"5min":1.216,"15min":1.244},"min":{"1min":0.90000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.603,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":0.77800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301976,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069864,"statfs":{"total":96636764160,"available":96327540736,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.876,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.077}]},{"osd":1,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1140000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.163}]},{"osd":2,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.286,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.103,"15min":1.129},"min":{"1min":0.879,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.389,"5min":1.72,"15min":1.72},"last":1.3149999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.02,"15min":1.042},"min":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2290000000000001,"5min":1.417,"15min":1.417},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0940000000000001,"15min":1.1240000000000001},"min":{"1min":0.78400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2969999999999999,"5min":1.593,"15min":1.593},"last":1.1120000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.028,"15min":1.075},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1659999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.117,"5min":1.161,"15min":1.1850000000000001},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3560000000000001,"5min":1.889,"15min":1.889},"last":1.3460000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.133,"15min":1.1739999999999999},"min":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3420000000000001,"5min":1.734,"15min":1.734},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.1839999999999999,"15min":1.23},"min":{"1min":0.86599999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.268,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.2769999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.3},"min":{"1min":1.004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4390000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.444},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3180000000000001,"15min":1.3600000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.4590000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.5389999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301992,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069848,"statfs":{"total":96636764160,"available":96327524352,"internally_reserved":0,"allocated":487424,"data_stored":200726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.875},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2450000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.626,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.747}]},{"osd":1,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.875},{"interface":"front","average":{"1min":1.048,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.749,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.94299999999999995},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.23,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.008},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93000000000000005,"15min":0.92200000000000004},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.93700000000000006,"15min":0.94299999999999995},"min":{"1min":0.71099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.181,"5min":1.794,"15min":1.794},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.048,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.95399999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0249999999999999,"15min":1.075},"min":{"1min":0.76100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.996,"5min":1.0609999999999999,"15min":1.095},"min":{"1min":0.79900000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.048}]},{"osd":6,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.075,"15min":1.101},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.333,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.504,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.111,"15min":1.135},"min":{"1min":0.90700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2470000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":1.038,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.587,"5min":1.99,"15min":1.99},"last":1.3380000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":475136,"data_stored":193599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.843,"15min":1.843},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.55700000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92400000000000004,"5min":1.202,"15min":1.202},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0129999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.65600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0289999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0129999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.997},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1859999999999999,"5min":1.956,"15min":1.956},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.071,"15min":1.083},"min":{"1min":0.92500000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.151,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.95199999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.81799999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.218,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.077,"15min":1.101},"min":{"1min":0.69399999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.196,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":0.83799999999999997}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0920000000000001,"15min":1.107},"min":{"1min":0.81299999999999994,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2390000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.86499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.1679999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.002}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.90900000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.222,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0920000000000001,"15min":1.0800000000000001},"min":{"1min":0.93000000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.337,"5min":1.573,"15min":1.573},"last":1.0669999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":479232,"data_stored":194547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.008,"15min":1.008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.54000000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.66500000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.70799999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.82099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.26,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.125},{"interface":"front","average":{"1min":1.054,"5min":1.022,"15min":1.012},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.448,"5min":1.734,"15min":1.734},"last":1.0329999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0189999999999999,"15min":0.997},"min":{"1min":0.72699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.008,"15min":1.018},"min":{"1min":0.79200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.173,"5min":1.234,"15min":1.234},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.339,"5min":1.375,"15min":1.375},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3089999999999999,"5min":1.397,"15min":1.397},"last":1.0589999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1180000000000001,"15min":1.1040000000000001},"min":{"1min":0.91200000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.98999999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.268,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.83499999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297816,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074024,"statfs":{"total":96636764160,"available":96331800576,"internally_reserved":0,"allocated":466944,"data_stored":193043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.123},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.22,"15min":1.22},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0960000000000001,"5min":1.167,"15min":1.167},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.69599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.004,"15min":1.018},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4890000000000001,"5min":1.647,"15min":1.647},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.76600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4510000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1000000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.248,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.073},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.071},"min":{"1min":0.94599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.85599999999999998}]},{"osd":6,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.075,"15min":1.095},"min":{"1min":0.81100000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.331,"5min":1.96,"15min":1.96},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.3580000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.2529999999999999},"min":{"1min":0.90600000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.4510000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.274,"5min":1.153,"15min":1.125},"min":{"1min":1.0469999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.123}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302556,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069284,"statfs":{"total":96636764160,"available":96326946816,"internally_reserved":0,"allocated":1056768,"data_stored":782414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.119,"15min":1.119},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0169999999999999,"5min":1.179,"15min":1.179},"last":0.495}]},{"osd":2,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.78700000000000003},"min":{"1min":0.752,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0660000000000001,"5min":1.385,"15min":1.385},"last":0.66300000000000003}]},{"osd":3,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.044,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.95899999999999996,"15min":0.94899999999999995},"min":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.252,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.304,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.377,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.071,"15min":1.0569999999999999},"min":{"1min":0.92200000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.327,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.044,"15min":1.038},"min":{"1min":0.84899999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.169,"15min":1.1679999999999999},"min":{"1min":1.0489999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.385,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.88900000000000001}]},{"osd":7,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.125},"min":{"1min":0.96299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.335,"5min":1.877,"15min":1.877},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":1.052,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4239999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.97299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:03:01.661 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-15T17:03:01.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 Processor -- start 2024-08-15T17:03:01.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- start start 2024-08-15T17:03:01.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff1080d0010 0x7ff1080d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff10815cea0 0x7ff10815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080cf190 con 0x7ff108159040 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1080d09a0 con 0x7ff10815cea0 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1080ce5e0 con 0x7ff1080d0010 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff10815cea0 0x7ff10815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10d8db700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7ff108159040 0x7ff1080cf620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41860/0 (socket says 172.21.15.146:41860) 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7ff10815cea0 0x7ff10815d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33506/0 (socket says 172.21.15.146:33506) 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10d8db700 1 -- 172.21.15.146:0/276737096 learned_addr learned my addr 172.21.15.146:0/276737096 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10d0da700 1 --2- 172.21.15.146:0/276737096 >> v2:172.21.15.161:3300/0 conn(0x7ff1080d0010 0x7ff1080d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 -- 172.21.15.146:0/276737096 >> v2:172.21.15.161:3300/0 conn(0x7ff1080d0010 msgr2=0x7ff1080d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 --2- 172.21.15.146:0/276737096 >> v2:172.21.15.161:3300/0 conn(0x7ff1080d0010 0x7ff1080d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 -- 172.21.15.146:0/276737096 >> v1:172.21.15.146:6789/0 conn(0x7ff108159040 legacy=0x7ff1080cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 -- 172.21.15.146:0/276737096 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080ce860 con 0x7ff10815cea0 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e0dc700 1 --2- 172.21.15.146:0/276737096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff10815cea0 0x7ff10815d2e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff108067ef0 tx=0x7ff100009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afd4e55ac506d43 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10c8d9700 1 -- 172.21.15.146:0/276737096 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000e070 con 0x7ff10815cea0 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10c8d9700 1 -- 172.21.15.146:0/276737096 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10000bc30 con 0x7ff10815cea0 2024-08-15T17:03:01.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10c8d9700 1 -- 172.21.15.146:0/276737096 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100012430 con 0x7ff10815cea0 2024-08-15T17:03:01.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- 172.21.15.146:0/276737096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff10815cea0 msgr2=0x7ff10815d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/276737096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff10815cea0 0x7ff10815d2e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff108067ef0 tx=0x7ff100009780 comp rx=0 tx=0).stop 2024-08-15T17:03:01.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- 172.21.15.146:0/276737096 shutdown_connections 2024-08-15T17:03:01.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/276737096 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff10815cea0 0x7ff10815d2e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/276737096 >> v2:172.21.15.161:3300/0 conn(0x7ff1080d0010 0x7ff1080d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.754+0000 7ff10e8dd700 1 -- 172.21.15.146:0/276737096 >> 172.21.15.146:0/276737096 conn(0x7ff108068250 msgr2=0x7ff108068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- 172.21.15.146:0/276737096 shutdown_connections 2024-08-15T17:03:01.770 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- 172.21.15.146:0/276737096 wait complete. 2024-08-15T17:03:01.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 Processor -- start 2024-08-15T17:03:01.772 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- start start 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff108159040 0x7ff1080a74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10815cea0 0x7ff108154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108169690 con 0x7ff1080d0010 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080a7e60 con 0x7ff10815cea0 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10e8dd700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff108169950 con 0x7ff108159040 2024-08-15T17:03:01.773 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10d8db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10d8db700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33510/0 (socket says 172.21.15.146:33510) 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10d8db700 1 -- 172.21.15.146:0/2659461636 learned_addr learned my addr 172.21.15.146:0/2659461636 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.758+0000 7ff10d0da700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff108159040 0x7ff1080a74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10e0dc700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10815cea0 0x7ff108154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 -- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff108159040 msgr2=0x7ff1080a74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff108159040 0x7ff1080a74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 -- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10815cea0 msgr2=0x7ff108154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.774 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10815cea0 0x7ff108154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 -- 172.21.15.146:0/2659461636 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009a20 con 0x7ff1080d0010 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10d8db700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c560 tx=0x7ff104001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104005b30 con 0x7ff1080d0010 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff104005c90 con 0x7ff1080d0010 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104007320 con 0x7ff1080d0010 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff108169c30 con 0x7ff1080d0010 2024-08-15T17:03:01.775 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff108152150 con 0x7ff1080d0010 2024-08-15T17:03:01.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1080643a0 con 0x7ff1080d0010 2024-08-15T17:03:01.776 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7ff10401b3b0 con 0x7ff1080d0010 2024-08-15T17:03:01.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 0x7ff0ec062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:01.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.762+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7ff10407bc60 con 0x7ff1080d0010 2024-08-15T17:03:01.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.766+0000 7ff10d0da700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 0x7ff0ec062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:01.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.766+0000 7ff10d0da700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 0x7ff0ec062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff108063a70 tx=0x7ff0f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:01.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.774+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7ff10404c350 con 0x7ff1080d0010 2024-08-15T17:03:01.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1080b3480 con 0x7ff1080d0010 2024-08-15T17:03:01.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff0fdffb700 1 -- 172.21.15.146:0/2659461636 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12443 (secure 0 0 0) 0x7ff10404f970 con 0x7ff1080d0010 2024-08-15T17:03:01.949 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:01.950 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":43,"fsid":"10958773-fd73-437e-aa56-f078e9b084bc","created":"2024-08-15T16:59:25.321118+0000","modified":"2024-08-15T17:02:58.585353+0000","last_up_change":"2024-08-15T16:59:35.356214+0000","last_in_change":"2024-08-15T16:59:27.358892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-15T16:59:35.707086+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"144fe671-bcc7-4fc6-bc5f-f0a3ccdfba0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":24267},{"type":"v1","addr":"172.21.15.146:6825","nonce":24267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":24267},{"type":"v1","addr":"172.21.15.146:6827","nonce":24267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":24267},{"type":"v1","addr":"172.21.15.146:6831","nonce":24267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":24267},{"type":"v1","addr":"172.21.15.146:6829","nonce":24267}]},"public_addr":"172.21.15.146:6825/24267","cluster_addr":"172.21.15.146:6827/24267","heartbeat_back_addr":"172.21.15.146:6831/24267","heartbeat_front_addr":"172.21.15.146:6829/24267","state":["exists","up"]},{"osd":1,"uuid":"c36dbee1-7d4f-4553-a18a-c1a70cfc250a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":24248},{"type":"v1","addr":"172.21.15.146:6804","nonce":24248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":24248},{"type":"v1","addr":"172.21.15.146:6808","nonce":24248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":24248},{"type":"v1","addr":"172.21.15.146:6815","nonce":24248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":24248},{"type":"v1","addr":"172.21.15.146:6812","nonce":24248}]},"public_addr":"172.21.15.146:6804/24248","cluster_addr":"172.21.15.146:6808/24248","heartbeat_back_addr":"172.21.15.146:6815/24248","heartbeat_front_addr":"172.21.15.146:6812/24248","state":["exists","up"]},{"osd":2,"uuid":"2a7a86d9-0c58-408b-b2e1-cce212321635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":24250},{"type":"v1","addr":"172.21.15.146:6817","nonce":24250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":24250},{"type":"v1","addr":"172.21.15.146:6819","nonce":24250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":24250},{"type":"v1","addr":"172.21.15.146:6823","nonce":24250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":24250},{"type":"v1","addr":"172.21.15.146:6821","nonce":24250}]},"public_addr":"172.21.15.146:6817/24250","cluster_addr":"172.21.15.146:6819/24250","heartbeat_back_addr":"172.21.15.146:6823/24250","heartbeat_front_addr":"172.21.15.146:6821/24250","state":["exists","up"]},{"osd":3,"uuid":"7171ad55-1f6b-440e-9ae7-b204c7596597","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":24249},{"type":"v1","addr":"172.21.15.146:6801","nonce":24249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6803","nonce":24249},{"type":"v1","addr":"172.21.15.146:6805","nonce":24249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6811","nonce":24249},{"type":"v1","addr":"172.21.15.146:6813","nonce":24249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6807","nonce":24249},{"type":"v1","addr":"172.21.15.146:6809","nonce":24249}]},"public_addr":"172.21.15.146:6801/24249","cluster_addr":"172.21.15.146:6805/24249","heartbeat_back_addr":"172.21.15.146:6813/24249","heartbeat_front_addr":"172.21.15.146:6809/24249","state":["exists","up"]},{"osd":4,"uuid":"b1084d0b-f46e-4452-b3d7-99809b0c8d1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":24465},{"type":"v1","addr":"172.21.15.161:6825","nonce":24465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":24465},{"type":"v1","addr":"172.21.15.161:6827","nonce":24465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":24465},{"type":"v1","addr":"172.21.15.161:6831","nonce":24465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":24465},{"type":"v1","addr":"172.21.15.161:6829","nonce":24465}]},"public_addr":"172.21.15.161:6825/24465","cluster_addr":"172.21.15.161:6827/24465","heartbeat_back_addr":"172.21.15.161:6831/24465","heartbeat_front_addr":"172.21.15.161:6829/24465","state":["exists","up"]},{"osd":5,"uuid":"0b2adc71-3d6d-4c6d-99bb-c8fe235ed7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":24464},{"type":"v1","addr":"172.21.15.161:6817","nonce":24464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":24464},{"type":"v1","addr":"172.21.15.161:6819","nonce":24464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":24464},{"type":"v1","addr":"172.21.15.161:6823","nonce":24464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":24464},{"type":"v1","addr":"172.21.15.161:6821","nonce":24464}]},"public_addr":"172.21.15.161:6817/24464","cluster_addr":"172.21.15.161:6819/24464","heartbeat_back_addr":"172.21.15.161:6823/24464","heartbeat_front_addr":"172.21.15.161:6821/24464","state":["exists","up"]},{"osd":6,"uuid":"edc57359-0752-4dfd-a5b2-1ee430f185d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":24462},{"type":"v1","addr":"172.21.15.161:6801","nonce":24462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":24462},{"type":"v1","addr":"172.21.15.161:6803","nonce":24462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":24462},{"type":"v1","addr":"172.21.15.161:6807","nonce":24462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":24462},{"type":"v1","addr":"172.21.15.161:6805","nonce":24462}]},"public_addr":"172.21.15.161:6801/24462","cluster_addr":"172.21.15.161:6803/24462","heartbeat_back_addr":"172.21.15.161:6807/24462","heartbeat_front_addr":"172.21.15.161:6805/24462","state":["exists","up"]},{"osd":7,"uuid":"4fb88d84-7e68-48c6-abac-c014247e680e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":24463},{"type":"v1","addr":"172.21.15.161:6809","nonce":24463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":24463},{"type":"v1","addr":"172.21.15.161:6811","nonce":24463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":24463},{"type":"v1","addr":"172.21.15.161:6815","nonce":24463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":24463},{"type":"v1","addr":"172.21.15.161:6813","nonce":24463}]},"public_addr":"172.21.15.161:6809/24463","cluster_addr":"172.21.15.161:6811/24463","heartbeat_back_addr":"172.21.15.161:6815/24463","heartbeat_front_addr":"172.21.15.161:6813/24463","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.876490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.335327+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.559755+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.569259+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.819579+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.907721+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.594496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-15T16:59:32.342593+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.146:6835/163298469":"2024-08-16T17:02:55.567258+0000","172.21.15.146:6834/163298469":"2024-08-16T17:02:55.567258+0000","172.21.15.161:6837/2559773181":"2024-08-16T17:02:49.326606+0000","172.21.15.161:6836/2559773181":"2024-08-16T17:02:49.326606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-15T17:03:01.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 msgr2=0x7ff0ec062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 0x7ff0ec062d10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff108063a70 tx=0x7ff0f8008040 comp rx=0 tx=0).stop 2024-08-15T17:03:01.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 msgr2=0x7ff108151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:01.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.938+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c560 tx=0x7ff104001020 comp rx=0 tx=0).stop 2024-08-15T17:03:01.952 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 shutdown_connections 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7ff0ec060860 0x7ff0ec062d10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff10815cea0 0x7ff108154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff108159040 0x7ff1080a74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 --2- 172.21.15.146:0/2659461636 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff1080d0010 0x7ff108151c10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 >> 172.21.15.146:0/2659461636 conn(0x7ff108068250 msgr2=0x7ff108152c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 shutdown_connections 2024-08-15T17:03:01.953 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:01.942+0000 7ff10e8dd700 1 -- 172.21.15.146:0/2659461636 wait complete. 2024-08-15T17:03:02.964 INFO:tasks.ceph:Scrubbing osd.0 2024-08-15T17:03:02.965 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.074+0000 7f96dae66700 1 Processor -- start 2024-08-15T17:03:03.087 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.074+0000 7f96dae66700 1 -- start start 2024-08-15T17:03:03.089 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40cd2b0 0x7f96d40cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f96d40ceb70 0x7f96d4162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d40cc480 con 0x7f96d40caeb0 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d4159720 con 0x7f96d40ceb70 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40cb870 con 0x7f96d40cd2b0 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d3fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f96d40caeb0 0x7f96d40ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41868/0 (socket says 172.21.15.146:41868) 2024-08-15T17:03:03.090 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d3fff700 1 -- 172.21.15.146:0/4268852119 learned_addr learned my addr 172.21.15.146:0/4268852119 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 --2- 172.21.15.146:0/4268852119 >> v2:172.21.15.146:3300/0 conn(0x7f96d40ceb70 0x7f96d4162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d8c02700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40cd2b0 0x7f96d40cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d37fe700 1 -- 172.21.15.146:0/4268852119 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430797396 0 0) 0x7f96d40cc480 con 0x7f96d40caeb0 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d37fe700 1 -- 172.21.15.146:0/4268852119 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003570 con 0x7f96d40caeb0 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 -- 172.21.15.146:0/4268852119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40cd2b0 msgr2=0x7f96d40cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40cd2b0 0x7f96d40cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.091 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 -- 172.21.15.146:0/4268852119 >> v1:172.21.15.146:6789/0 conn(0x7f96d40caeb0 legacy=0x7f96d40ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 -- 172.21.15.146:0/4268852119 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40cbb50 con 0x7f96d40ceb70 2024-08-15T17:03:03.092 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d9403700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d4162ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e60 tx=0x7f96cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cec6e81f93195 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d37fe700 1 -- 172.21.15.146:0/4268852119 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00e070 con 0x7f96d40ceb70 2024-08-15T17:03:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d37fe700 1 -- 172.21.15.146:0/4268852119 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00bab0 con 0x7f96d40ceb70 2024-08-15T17:03:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96d37fe700 1 -- 172.21.15.146:0/4268852119 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00be30 con 0x7f96d40ceb70 2024-08-15T17:03:03.095 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- 172.21.15.146:0/4268852119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 msgr2=0x7f96d4162ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d4162ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e60 tx=0x7f96cc009780 comp rx=0 tx=0).stop 2024-08-15T17:03:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- 172.21.15.146:0/4268852119 shutdown_connections 2024-08-15T17:03:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d4162ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 --2- 172.21.15.146:0/4268852119 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40cd2b0 0x7f96d40cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.096 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- 172.21.15.146:0/4268852119 >> 172.21.15.146:0/4268852119 conn(0x7f96d4099550 msgr2=0x7f96d4067590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- 172.21.15.146:0/4268852119 shutdown_connections 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.078+0000 7f96dae66700 1 -- 172.21.15.146:0/4268852119 wait complete. 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 Processor -- start 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- start start 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40caeb0 0x7f96d40c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96d40cd2b0 0x7f96d414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d40b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.097 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b30f0 con 0x7f96d40ceb70 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b5f30 con 0x7f96d40cd2b0 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40c8330 con 0x7f96d40caeb0 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d40b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d40b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33522/0 (socket says 172.21.15.146:33522) 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 -- 172.21.15.146:0/3784752982 learned_addr learned my addr 172.21.15.146:0/3784752982 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d8c02700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40caeb0 0x7f96d40c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.098 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d3fff700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96d40cd2b0 0x7f96d414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 -- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40caeb0 msgr2=0x7f96d40c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96d40caeb0 0x7f96d40c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 -- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96d40cd2b0 msgr2=0x7f96d414ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96d40cd2b0 0x7f96d414ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009a20 con 0x7f96d40ceb70 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d9403700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d40b59f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f96cc013000 tx=0x7f96cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.099 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc012c00 con 0x7f96d40ceb70 2024-08-15T17:03:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc021070 con 0x7f96d40ceb70 2024-08-15T17:03:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d40c8610 con 0x7f96d40ceb70 2024-08-15T17:03:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc026430 con 0x7f96d40ceb70 2024-08-15T17:03:03.100 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.082+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d40c88f0 con 0x7f96d40ceb70 2024-08-15T17:03:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f96d40638f0 con 0x7f96d40ceb70 2024-08-15T17:03:03.101 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f96cc023030 con 0x7f96d40ceb70 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f96c0060860 0x7f96c0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f96cc07e9a0 con 0x7f96d40ceb70 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d8c02700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f96c0060860 0x7f96c0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f96c0066750 0x7f96c0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96c0069210 con 0x7f96c0066750 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96cc083220 con 0x7f96d40ceb70 2024-08-15T17:03:03.102 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.086+0000 7f96d3fff700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f96c0066750 0x7f96c0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.090+0000 7f96d8c02700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f96c0060860 0x7f96c0062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009d10 tx=0x7f96c80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.090+0000 7f96d3fff700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f96c0066750 0x7f96c0068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.103 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.090+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== osd.0 v2:172.21.15.146:6824/24267 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f96c0069210 con 0x7f96c0066750 2024-08-15T17:03:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 --> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f96d4068020 con 0x7f96c0066750 2024-08-15T17:03:03.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96d0ff9700 1 -- 172.21.15.146:0/3784752982 <== osd.0 v2:172.21.15.146:6824/24267 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f96d4068020 con 0x7f96c0066750 2024-08-15T17:03:03.117 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:03.117 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:03.117 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f96c0066750 msgr2=0x7f96c0068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:6824/24267,v1:172.21.15.146:6825/24267] conn(0x7f96c0066750 0x7f96c0068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f96c0060860 msgr2=0x7f96c0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f96c0060860 0x7f96c0062d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009d10 tx=0x7f96c80091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 msgr2=0x7f96d40b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 --2- 172.21.15.146:0/3784752982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96d40ceb70 0x7f96d40b59f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f96cc013000 tx=0x7f96cc019000 comp rx=0 tx=0).stop 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96d9403700 1 -- 172.21.15.146:0/3784752982 reap_dead start 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 shutdown_connections 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 >> 172.21.15.146:0/3784752982 conn(0x7f96d4099550 msgr2=0x7f96d40c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.119 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 shutdown_connections 2024-08-15T17:03:03.120 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.106+0000 7f96dae66700 1 -- 172.21.15.146:0/3784752982 wait complete. 2024-08-15T17:03:03.130 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-15T17:03:03.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.230+0000 7f228e27a700 1 Processor -- start 2024-08-15T17:03:03.243 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.230+0000 7f228e27a700 1 -- start start 2024-08-15T17:03:03.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f228815a620 0x7f22880ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22880b0f10 0x7f22880adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22880ae530 con 0x7f22880ae0d0 2024-08-15T17:03:03.244 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880aff10 con 0x7f22880b0f10 2024-08-15T17:03:03.245 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880a9160 con 0x7f228815a620 2024-08-15T17:03:03.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f22880b0f10 0x7f22880adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228c817700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f22880ae0d0 0x7f228806e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41872/0 (socket says 172.21.15.146:41872) 2024-08-15T17:03:03.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228c817700 1 -- 172.21.15.146:0/1195568909 learned_addr learned my addr 172.21.15.146:0/1195568909 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:03.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f2287fff700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f228815a620 0x7f22880ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 -- 172.21.15.146:0/1195568909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f228815a620 msgr2=0x7f22880ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f228815a620 0x7f22880ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.248 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 -- 172.21.15.146:0/1195568909 >> v1:172.21.15.146:6789/0 conn(0x7f22880ae0d0 legacy=0x7f228806e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f2286ffd700 1 -- 172.21.15.146:0/1195568909 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809914026 0 0) 0x7f2280001490 con 0x7f22880ae0d0 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 -- 172.21.15.146:0/1195568909 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880b1680 con 0x7f22880b0f10 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f22877fe700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22880b0f10 0x7f22880adb90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f22780098e0 tx=0x7f22780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664efbb37da34d93 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f2286ffd700 1 -- 172.21.15.146:0/1195568909 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800b640 con 0x7f22880b0f10 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f2286ffd700 1 -- 172.21.15.146:0/1195568909 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227800b7a0 con 0x7f22880b0f10 2024-08-15T17:03:03.249 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f2286ffd700 1 -- 172.21.15.146:0/1195568909 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800bb40 con 0x7f22880b0f10 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- 172.21.15.146:0/1195568909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22880b0f10 msgr2=0x7f22880adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22880b0f10 0x7f22880adb90 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f22780098e0 tx=0x7f22780091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- 172.21.15.146:0/1195568909 shutdown_connections 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f22880b0f10 0x7f22880adb90 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 --2- 172.21.15.146:0/1195568909 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f228815a620 0x7f22880ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.250 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- 172.21.15.146:0/1195568909 >> 172.21.15.146:0/1195568909 conn(0x7f2288067810 msgr2=0x7f2288067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- 172.21.15.146:0/1195568909 shutdown_connections 2024-08-15T17:03:03.251 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.234+0000 7f228e27a700 1 -- 172.21.15.146:0/1195568909 wait complete. 2024-08-15T17:03:03.253 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 Processor -- start 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- start start 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22880ae0d0 0x7f2288155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22880b0f10 0x7f22880cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880ab980 con 0x7f228815a620 2024-08-15T17:03:03.254 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880abf00 con 0x7f22880ae0d0 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2288156430 con 0x7f22880b0f10 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f22877fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22880b0f10 0x7f22880cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33548/0 (socket says 172.21.15.146:33548) 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 -- 172.21.15.146:0/2962777578 learned_addr learned my addr 172.21.15.146:0/2962777578 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.255 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f2287fff700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22880ae0d0 0x7f2288155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 -- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22880b0f10 msgr2=0x7f22880cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22880b0f10 0x7f22880cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 -- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22880ae0d0 msgr2=0x7f2288155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22880ae0d0 0x7f2288155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 -- 172.21.15.146:0/2962777578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278009420 con 0x7f228815a620 2024-08-15T17:03:03.256 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228c817700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f228000cbf0 tx=0x7f22800019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000a8b0 con 0x7f228815a620 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228000aa10 con 0x7f228815a620 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22880ca330 con 0x7f228815a620 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228000add0 con 0x7f228815a620 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.238+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22880abb90 con 0x7f228815a620 2024-08-15T17:03:03.257 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22880ac070 con 0x7f228815a620 2024-08-15T17:03:03.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f2280007300 con 0x7f228815a620 2024-08-15T17:03:03.258 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f227ffff700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 0x7f2268062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f228007ca40 con 0x7f228815a620 2024-08-15T17:03:03.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f2287fff700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 0x7f2268062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.259 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.242+0000 7f2287fff700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 0x7f2268062d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2270009db0 tx=0x7f22700091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.263 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.254+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f228007ee60 con 0x7f228815a620 2024-08-15T17:03:03.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f22880645c0 con 0x7f2268060860 2024-08-15T17:03:03.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f227ffff700 1 -- 172.21.15.146:0/2962777578 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f22880645c0 con 0x7f2268060860 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 msgr2=0x7f2268062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 0x7f2268062d10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2270009db0 tx=0x7f22700091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 msgr2=0x7f22880c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f228000cbf0 tx=0x7f22800019b0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 shutdown_connections 2024-08-15T17:03:03.435 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f2268060860 0x7f2268062d10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f228815a620 0x7f22880c71c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22880b0f10 0x7f22880cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 --2- 172.21.15.146:0/2962777578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22880ae0d0 0x7f2288155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 >> 172.21.15.146:0/2962777578 conn(0x7f2288067810 msgr2=0x7f228806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 shutdown_connections 2024-08-15T17:03:03.436 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.422+0000 7f228e27a700 1 -- 172.21.15.146:0/2962777578 wait complete. 2024-08-15T17:03:03.437 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 0 to deep-scrub 2024-08-15T17:03:03.446 INFO:tasks.ceph:Scrubbing osd.1 2024-08-15T17:03:03.446 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:03.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.550+0000 7f49d5d20700 1 Processor -- start 2024-08-15T17:03:03.563 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.550+0000 7f49d5d20700 1 -- start start 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d0142960 0x7f49d00b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49d00b1330 0x7f49d00687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 0x7f49d00b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00aeb20 con 0x7f49d00b1330 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d00b1780 con 0x7f49d0068ce0 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d5d20700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d00694b0 con 0x7f49d0142960 2024-08-15T17:03:03.567 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49d00b1330 0x7f49d00687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f49d00b1330 0x7f49d00687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33560/0 (socket says 172.21.15.146:33560) 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 -- 172.21.15.146:0/3869408647 learned_addr learned my addr 172.21.15.146:0/3869408647 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d551f700 1 --2- 172.21.15.146:0/3869408647 >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 0x7f49d00b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d4d1e700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d0142960 0x7f49d00b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 -- 172.21.15.146:0/3869408647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d0142960 msgr2=0x7f49d00b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d0142960 0x7f49d00b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.568 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 -- 172.21.15.146:0/3869408647 >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 msgr2=0x7f49d00b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 --2- 172.21.15.146:0/3869408647 >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 0x7f49d00b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 -- 172.21.15.146:0/3869408647 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0069790 con 0x7f49d00b1330 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49d551f700 1 --2- 172.21.15.146:0/3869408647 >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 0x7f49d00b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c7fff700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d00b1330 0x7f49d00687a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f49d00cdeb0 tx=0x7f49c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ff7cf16cca6f0c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c77fe700 1 -- 172.21.15.146:0/3869408647 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000b640 con 0x7f49d00b1330 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c77fe700 1 -- 172.21.15.146:0/3869408647 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000b7a0 con 0x7f49d00b1330 2024-08-15T17:03:03.569 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.554+0000 7f49c77fe700 1 -- 172.21.15.146:0/3869408647 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000bb60 con 0x7f49d00b1330 2024-08-15T17:03:03.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- 172.21.15.146:0/3869408647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d00b1330 msgr2=0x7f49d00687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d00b1330 0x7f49d00687a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f49d00cdeb0 tx=0x7f49c0009250 comp rx=0 tx=0).stop 2024-08-15T17:03:03.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- 172.21.15.146:0/3869408647 shutdown_connections 2024-08-15T17:03:03.572 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- 172.21.15.146:0/3869408647 >> v2:172.21.15.161:3300/0 conn(0x7f49d0068ce0 0x7f49d00b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d00b1330 0x7f49d00687a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- 172.21.15.146:0/3869408647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d0142960 0x7f49d00b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- 172.21.15.146:0/3869408647 >> 172.21.15.146:0/3869408647 conn(0x7f49d006cd20 msgr2=0x7f49d006d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- 172.21.15.146:0/3869408647 shutdown_connections 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- 172.21.15.146:0/3869408647 wait complete. 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 Processor -- start 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- start start 2024-08-15T17:03:03.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d0068ce0 0x7f49d0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d00b1330 0x7f49d015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 0x7f49d013dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015fec0 con 0x7f49d0142960 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d013fd70 con 0x7f49d0068ce0 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d5d20700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d015cc80 con 0x7f49d00b1330 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d551f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 0x7f49d013dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.574 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49c7fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d00b1330 0x7f49d015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d551f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 0x7f49d013dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33564/0 (socket says 172.21.15.146:33564) 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d551f700 1 -- 172.21.15.146:0/4004261079 learned_addr learned my addr 172.21.15.146:0/4004261079 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.558+0000 7f49d4d1e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d0068ce0 0x7f49d0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 -- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d00b1330 msgr2=0x7f49d015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f49d00b1330 0x7f49d015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 -- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d0068ce0 msgr2=0x7f49d0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d0068ce0 0x7f49d0158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8009a20 con 0x7f49d0142960 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d4d1e700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f49d0068ce0 0x7f49d0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d551f700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 0x7f49d013dc90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009fc0 tx=0x7f49c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c800cbd0 con 0x7f49d0142960 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c800f040 con 0x7f49d0142960 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00094f0 con 0x7f49d0142960 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8014480 con 0x7f49d0142960 2024-08-15T17:03:03.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d016d970 con 0x7f49d0142960 2024-08-15T17:03:03.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49d00644b0 con 0x7f49d0142960 2024-08-15T17:03:03.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.562+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f49c8003020 con 0x7f49d0142960 2024-08-15T17:03:03.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f49bc060860 0x7f49bc062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f49c807d1c0 con 0x7f49d0142960 2024-08-15T17:03:03.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7f49bc066750 0x7f49bc068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49bc069210 con 0x7f49bc066750 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49d4d1e700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f49bc060860 0x7f49bc062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c807f800 con 0x7f49d0142960 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c7fff700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7f49bc066750 0x7f49bc068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c7fff700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7f49bc066750 0x7f49bc068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49d4d1e700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f49bc060860 0x7f49bc062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0113e0 tx=0x7f49cc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.566+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== osd.1 v2:172.21.15.146:6802/24248 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f49bc069210 con 0x7f49bc066750 2024-08-15T17:03:03.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 --> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f49d00b1930 con 0x7f49bc066750 2024-08-15T17:03:03.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49c4ff9700 1 -- 172.21.15.146:0/4004261079 <== osd.1 v2:172.21.15.146:6802/24248 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f49d00b1930 con 0x7f49bc066750 2024-08-15T17:03:03.592 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:03.592 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:03.593 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:03.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7f49bc066750 msgr2=0x7f49bc068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:6802/24248,v1:172.21.15.146:6804/24248] conn(0x7f49bc066750 0x7f49bc068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f49bc060860 msgr2=0x7f49bc062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f49bc060860 0x7f49bc062d10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0113e0 tx=0x7f49cc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 msgr2=0x7f49d013dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 --2- 172.21.15.146:0/4004261079 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f49d0142960 0x7f49d013dc90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009fc0 tx=0x7f49c800b040 comp rx=0 tx=0).stop 2024-08-15T17:03:03.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d551f700 1 -- 172.21.15.146:0/4004261079 reap_dead start 2024-08-15T17:03:03.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 shutdown_connections 2024-08-15T17:03:03.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 >> 172.21.15.146:0/4004261079 conn(0x7f49d006cd20 msgr2=0x7f49d015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 shutdown_connections 2024-08-15T17:03:03.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.582+0000 7f49d5d20700 1 -- 172.21.15.146:0/4004261079 wait complete. 2024-08-15T17:03:03.604 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-15T17:03:03.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.706+0000 7fd4ec9e3700 1 Processor -- start 2024-08-15T17:03:03.718 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.706+0000 7fd4ec9e3700 1 -- start start 2024-08-15T17:03:03.720 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 0x7fd4e80c8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd4e8158000 0x7fd4e80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 0x7fd4e80c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80ad670 con 0x7fd4e8158000 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e80acd80 con 0x7fd4e8157070 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8157440 con 0x7fd4e80ae040 2024-08-15T17:03:03.721 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd4e8158000 0x7fd4e80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e7fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 0x7fd4e80c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fd4e8158000 0x7fd4e80adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33566/0 (socket says 172.21.15.146:33566) 2024-08-15T17:03:03.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e77fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 0x7fd4e80c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.722 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 -- 172.21.15.146:0/3837743288 learned_addr learned my addr 172.21.15.146:0/3837743288 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 -- 172.21.15.146:0/3837743288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 msgr2=0x7fd4e80c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 0x7fd4e80c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 -- 172.21.15.146:0/3837743288 >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 msgr2=0x7fd4e80c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 --2- 172.21.15.146:0/3837743288 >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 0x7fd4e80c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 -- 172.21.15.146:0/3837743288 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e81585d0 con 0x7fd4e8158000 2024-08-15T17:03:03.724 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e77fe700 1 --2- 172.21.15.146:0/3837743288 >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 0x7fd4e80c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e7fff700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 0x7fd4e80c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4deffd700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8158000 0x7fd4e80adb00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8070620 tx=0x7fd4d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e207457e8ead8c0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e6ffd700 1 -- 172.21.15.146:0/3837743288 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000b640 con 0x7fd4e8158000 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e6ffd700 1 -- 172.21.15.146:0/3837743288 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d000b7a0 con 0x7fd4e8158000 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4e6ffd700 1 -- 172.21.15.146:0/3837743288 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000bb60 con 0x7fd4e8158000 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/3837743288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8158000 msgr2=0x7fd4e80adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.710+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8158000 0x7fd4e80adb00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8070620 tx=0x7fd4d0009250 comp rx=0 tx=0).stop 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/3837743288 shutdown_connections 2024-08-15T17:03:03.725 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e80ae040 0x7fd4e80c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/3837743288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8158000 0x7fd4e80adb00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/3837743288 >> v2:172.21.15.161:3300/0 conn(0x7fd4e8157070 0x7fd4e80c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/3837743288 >> 172.21.15.146:0/3837743288 conn(0x7fd4e80ccfc0 msgr2=0x7fd4e80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/3837743288 shutdown_connections 2024-08-15T17:03:03.726 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/3837743288 wait complete. 2024-08-15T17:03:03.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 Processor -- start 2024-08-15T17:03:03.727 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.714+0000 7fd4ec9e3700 1 -- start start 2024-08-15T17:03:03.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4e80ae040 0x7fd4e8143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.729 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e8158000 0x7fd4e815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8155700 con 0x7fd4e8157070 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8144500 con 0x7fd4e80ae040 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8155b90 con 0x7fd4e8158000 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33578/0 (socket says 172.21.15.146:33578) 2024-08-15T17:03:03.730 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 -- 172.21.15.146:0/2929705067 learned_addr learned my addr 172.21.15.146:0/2929705067 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4e7fff700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e8158000 0x7fd4e815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4e77fe700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4e80ae040 0x7fd4e8143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 -- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e8158000 msgr2=0x7fd4e815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e8158000 0x7fd4e815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 -- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4e80ae040 msgr2=0x7fd4e8143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4e80ae040 0x7fd4e8143fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 -- 172.21.15.146:0/2929705067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009a20 con 0x7fd4e8157070 2024-08-15T17:03:03.732 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4deffd700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8069330 tx=0x7fd4d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0000d10 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d000bb20 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d8009420 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001a6a0 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.718+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d00094f0 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e8155870 con 0x7fd4e8157070 2024-08-15T17:03:03.733 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fd4d001aad0 con 0x7fd4e8157070 2024-08-15T17:03:03.734 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4dffff700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 0x7fd4c8062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:03.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fd4d007e8f0 con 0x7fd4e8157070 2024-08-15T17:03:03.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4e77fe700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 0x7fd4c8062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:03.735 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.722+0000 7fd4e77fe700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 0x7fd4c8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009f40 tx=0x7fd4d80091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:03.736 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.726+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fd4d004f0a0 con 0x7fd4e8157070 2024-08-15T17:03:03.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.890+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4e80638f0 con 0x7fd4c8060860 2024-08-15T17:03:03.901 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.890+0000 7fd4dffff700 1 -- 172.21.15.146:0/2929705067 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd4e80638f0 con 0x7fd4c8060860 2024-08-15T17:03:03.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 msgr2=0x7fd4c8062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.905 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 0x7fd4c8062d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009f40 tx=0x7fd4d80091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 msgr2=0x7fd4e815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8069330 tx=0x7fd4d0017000 comp rx=0 tx=0).stop 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 shutdown_connections 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fd4c8060860 0x7fd4c8062d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4e8158000 0x7fd4e815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4e8157070 0x7fd4e815a990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.906 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 --2- 172.21.15.146:0/2929705067 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4e80ae040 0x7fd4e8143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:03.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 >> 172.21.15.146:0/2929705067 conn(0x7fd4e80ccfc0 msgr2=0x7fd4e8067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:03.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 shutdown_connections 2024-08-15T17:03:03.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:03.894+0000 7fd4ec9e3700 1 -- 172.21.15.146:0/2929705067 wait complete. 2024-08-15T17:03:03.907 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 1 to deep-scrub 2024-08-15T17:03:03.921 INFO:tasks.ceph:Scrubbing osd.2 2024-08-15T17:03:03.922 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:04.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.022+0000 7fbbe0fec700 1 Processor -- start 2024-08-15T17:03:04.035 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.022+0000 7fbbe0fec700 1 -- start start 2024-08-15T17:03:04.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbe0fec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbe0fec700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbdc0cea10 0x7fbbdc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbe0fec700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbdc0c5bb0 con 0x7fbbdc0cc230 2024-08-15T17:03:04.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbe0fec700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0cc670 con 0x7fbbdc0cea10 2024-08-15T17:03:04.037 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbe0fec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0c4fa0 con 0x7fbbdc0ce640 2024-08-15T17:03:04.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbdc0cea10 0x7fbbdc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdb7fe700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdbfff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fbbdc0cc230 0x7fbbdc0c6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:58306/0 (socket says 172.21.15.146:58306) 2024-08-15T17:03:04.038 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdbfff700 1 -- 172.21.15.146:0/2135858151 learned_addr learned my addr 172.21.15.146:0/2135858151 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:04.039 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 -- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 msgr2=0x7fbbdc0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 --2- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 -- 172.21.15.146:0/2135858151 >> v1:172.21.15.161:6789/0 conn(0x7fbbdc0cc230 legacy=0x7fbbdc0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 -- 172.21.15.146:0/2135858151 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0c5280 con 0x7fbbdc0cea10 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbd2ffd700 1 --2- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc0cbcf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098e0 tx=0x7fbbc40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8124321e5a1e26ae server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdaffd700 1 -- 172.21.15.146:0/2135858151 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400b640 con 0x7fbbdc0cea10 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdaffd700 1 -- 172.21.15.146:0/2135858151 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc400b7a0 con 0x7fbbdc0cea10 2024-08-15T17:03:04.040 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.026+0000 7fbbdaffd700 1 -- 172.21.15.146:0/2135858151 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400bb40 con 0x7fbbdc0cea10 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cea10 msgr2=0x7fbbdc0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc0cbcf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbbc40098e0 tx=0x7fbbc40091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- 172.21.15.146:0/2135858151 shutdown_connections 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc0cbcf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/2135858151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- 172.21.15.146:0/2135858151 >> 172.21.15.146:0/2135858151 conn(0x7fbbdc099410 msgr2=0x7fbbdc0675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- 172.21.15.146:0/2135858151 shutdown_connections 2024-08-15T17:03:04.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- 172.21.15.146:0/2135858151 wait complete. 2024-08-15T17:03:04.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 Processor -- start 2024-08-15T17:03:04.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 -- start start 2024-08-15T17:03:04.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.030+0000 7fbbe0fec700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cc230 0x7fbbdc1554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0c7180 con 0x7fbbdc0cea10 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0c7fe0 con 0x7fbbdc0cc230 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc1559f0 con 0x7fbbdc0ce640 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33592/0 (socket says 172.21.15.146:33592) 2024-08-15T17:03:04.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbd2ffd700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 -- 172.21.15.146:0/177010561 learned_addr learned my addr 172.21.15.146:0/177010561 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdb7fe700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cc230 0x7fbbdc1554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 -- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 msgr2=0x7fbbdc0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbdc0ce640 0x7fbbdc0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 -- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cc230 msgr2=0x7fbbdc1554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbdc0cc230 0x7fbbdc1554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4009420 con 0x7fbbdc0cea10 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbdbfff700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc156f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400cc70 tx=0x7fbbd4001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400a850 con 0x7fbbdc0cea10 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400a9b0 con 0x7fbbdc0cea10 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc0c73d0 con 0x7fbbdc0cea10 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400ad70 con 0x7fbbdc0cea10 2024-08-15T17:03:04.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc0c7990 con 0x7fbbdc0cea10 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fbbd4007300 con 0x7fbbdc0cea10 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.034+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbdc0644b0 con 0x7fbbdc0cea10 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbbc0060860 0x7fbbc0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fbbd407cc90 con 0x7fbbdc0cea10 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7fbbc0066750 0x7fbbc0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbc0069210 con 0x7fbbc0066750 2024-08-15T17:03:04.049 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbd407f2d0 con 0x7fbbdc0cea10 2024-08-15T17:03:04.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbdb7fe700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbbc0060860 0x7fbbc0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd2ffd700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7fbbc0066750 0x7fbbc0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd2ffd700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7fbbc0066750 0x7fbbc0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbdb7fe700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbbc0060860 0x7fbbc0062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009db0 tx=0x7fbbcc0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.050 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.038+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== osd.2 v2:172.21.15.146:6816/24250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbbc0069210 con 0x7fbbc0066750 2024-08-15T17:03:04.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 --> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbbdc0c7d30 con 0x7fbbc0066750 2024-08-15T17:03:04.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbd3fff700 1 -- 172.21.15.146:0/177010561 <== osd.2 v2:172.21.15.146:6816/24250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbbdc0c7d30 con 0x7fbbc0066750 2024-08-15T17:03:04.067 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:04.067 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:04.067 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:04.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7fbbc0066750 msgr2=0x7fbbc0068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:6816/24250,v1:172.21.15.146:6817/24250] conn(0x7fbbc0066750 0x7fbbc0068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbbc0060860 msgr2=0x7fbbc0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fbbc0060860 0x7fbbc0062d10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009db0 tx=0x7fbbcc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 msgr2=0x7fbbdc156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.054+0000 7fbbe0fec700 1 --2- 172.21.15.146:0/177010561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbdc0cea10 0x7fbbdc156f50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400cc70 tx=0x7fbbd4001a50 comp rx=0 tx=0).stop 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.058+0000 7fbbdbfff700 1 -- 172.21.15.146:0/177010561 reap_dead start 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.058+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 shutdown_connections 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.058+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 >> 172.21.15.146:0/177010561 conn(0x7fbbdc099410 msgr2=0x7fbbdc0699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.058+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 shutdown_connections 2024-08-15T17:03:04.069 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.058+0000 7fbbe0fec700 1 -- 172.21.15.146:0/177010561 wait complete. 2024-08-15T17:03:04.079 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-15T17:03:04.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.182+0000 7fc602476700 1 Processor -- start 2024-08-15T17:03:04.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.182+0000 7fc602476700 1 -- start start 2024-08-15T17:03:04.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc155d20 0x7fc5fc0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc0af2b0 con 0x7fc5fc06df10 2024-08-15T17:03:04.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc0adc10 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc0af590 con 0x7fc5fc155d20 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33596/0 (socket says 172.21.15.146:33596) 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 -- 172.21.15.146:0/1823728528 learned_addr learned my addr 172.21.15.146:0/1823728528 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc601474700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc155d20 0x7fc5fc0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 -- 172.21.15.146:0/1823728528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc155d20 msgr2=0x7fc5fc0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.198 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc155d20 0x7fc5fc0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 -- 172.21.15.146:0/1823728528 >> v1:172.21.15.161:6789/0 conn(0x7fc5fc06df10 legacy=0x7fc5fc1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 -- 172.21.15.146:0/1823728528 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc0af870 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc600c73700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80099b0 tx=0x7fc5e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc800e08a4684f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc5fbfff700 1 -- 172.21.15.146:0/1823728528 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8009ea0 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc5fbfff700 1 -- 172.21.15.146:0/1823728528 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800b860 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.199 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc5fbfff700 1 -- 172.21.15.146:0/1823728528 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800bc00 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- 172.21.15.146:0/1823728528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 msgr2=0x7fc5fc0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80099b0 tx=0x7fc5e8009250 comp rx=0 tx=0).stop 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- 172.21.15.146:0/1823728528 shutdown_connections 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc0aea80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 --2- 172.21.15.146:0/1823728528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc155d20 0x7fc5fc0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- 172.21.15.146:0/1823728528 >> 172.21.15.146:0/1823728528 conn(0x7fc5fc0678d0 msgr2=0x7fc5fc067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- 172.21.15.146:0/1823728528 shutdown_connections 2024-08-15T17:03:04.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.186+0000 7fc602476700 1 -- 172.21.15.146:0/1823728528 wait complete. 2024-08-15T17:03:04.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 Processor -- start 2024-08-15T17:03:04.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 -- start start 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc06df10 0x7fc5fc134ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5fc155d20 0x7fc5fc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc149290 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc149ac0 con 0x7fc5fc155d20 2024-08-15T17:03:04.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc602476700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc151040 con 0x7fc5fc06df10 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33610/0 (socket says 172.21.15.146:33610) 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 -- 172.21.15.146:0/992313000 learned_addr learned my addr 172.21.15.146:0/992313000 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc601474700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc06df10 0x7fc5fc134ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc601c75700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5fc155d20 0x7fc5fc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 -- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc06df10 msgr2=0x7fc5fc134ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc06df10 0x7fc5fc134ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 -- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5fc155d20 msgr2=0x7fc5fc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5fc155d20 0x7fc5fc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 -- 172.21.15.146:0/992313000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e80094f0 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc600c73700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06c2d0 tx=0x7fc5e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.190+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8000d10 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800bbe0 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc149470 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8011670 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc150a10 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc149c30 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc5e8011aa0 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc5f97fa700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 0x7fc5e4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.194+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fc5e807e150 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.198+0000 7fc601474700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 0x7fc5e4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.198+0000 7fc601474700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 0x7fc5e4062d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc14e5c0 tx=0x7fc5ec0091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.202+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc5e804e7a0 con 0x7fc5fc0ad7c0 2024-08-15T17:03:04.376 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.366+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5fc0643a0 con 0x7fc5e4060890 2024-08-15T17:03:04.377 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.366+0000 7fc5f97fa700 1 -- 172.21.15.146:0/992313000 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc5fc0643a0 con 0x7fc5e4060890 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 msgr2=0x7fc5e4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 0x7fc5e4062d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc14e5c0 tx=0x7fc5ec0091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 msgr2=0x7fc5fc14ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06c2d0 tx=0x7fc5e8017000 comp rx=0 tx=0).stop 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 shutdown_connections 2024-08-15T17:03:04.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5e4060890 0x7fc5e4062d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5fc155d20 0x7fc5fc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc5fc0ad7c0 0x7fc5fc14ca20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 --2- 172.21.15.146:0/992313000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5fc06df10 0x7fc5fc134ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 >> 172.21.15.146:0/992313000 conn(0x7fc5fc0678d0 msgr2=0x7fc5fc06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 shutdown_connections 2024-08-15T17:03:04.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.370+0000 7fc602476700 1 -- 172.21.15.146:0/992313000 wait complete. 2024-08-15T17:03:04.382 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 2 to deep-scrub 2024-08-15T17:03:04.391 INFO:tasks.ceph:Scrubbing osd.3 2024-08-15T17:03:04.391 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:04.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.494+0000 7fe4c9ff1700 1 Processor -- start 2024-08-15T17:03:04.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.494+0000 7fe4c9ff1700 1 -- start start 2024-08-15T17:03:04.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c9ff1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c406e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c9ff1700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4c40686a0 0x7fe4c4068b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c9ff1700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c40abd70 con 0x7fe4c4155390 2024-08-15T17:03:04.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c9ff1700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c40af170 con 0x7fe4c40686a0 2024-08-15T17:03:04.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c9ff1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4069060 con 0x7fe4c4155760 2024-08-15T17:03:04.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4c40686a0 0x7fe4c4068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c3fff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c406e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4c40686a0 0x7fe4c4068b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:43042/0 (socket says 172.21.15.146:43042) 2024-08-15T17:03:04.511 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 -- 172.21.15.146:0/1562894100 learned_addr learned my addr 172.21.15.146:0/1562894100 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 -- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 msgr2=0x7fe4c406e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 --2- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c406e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 -- 172.21.15.146:0/1562894100 >> v1:172.21.15.161:6789/0 conn(0x7fe4c4155390 legacy=0x7fe4c40ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 -- 172.21.15.146:0/1562894100 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40ac050 con 0x7fe4c40686a0 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c97f0700 1 --2- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c40686a0 0x7fe4c4068b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c0d0 tx=0x7fe4bc00a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a5b3a3f8d53c66 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c37fe700 1 -- 172.21.15.146:0/1562894100 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00f070 con 0x7fe4c40686a0 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c37fe700 1 -- 172.21.15.146:0/1562894100 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc00a2e0 con 0x7fe4c40686a0 2024-08-15T17:03:04.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.498+0000 7fe4c37fe700 1 -- 172.21.15.146:0/1562894100 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00cd40 con 0x7fe4c40686a0 2024-08-15T17:03:04.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c40686a0 msgr2=0x7fe4c4068b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c40686a0 0x7fe4c4068b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c0d0 tx=0x7fe4bc00a950 comp rx=0 tx=0).stop 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/1562894100 shutdown_connections 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c40686a0 0x7fe4c4068b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/1562894100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c406e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/1562894100 >> 172.21.15.146:0/1562894100 conn(0x7fe4c40ccfe0 msgr2=0x7fe4c406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/1562894100 shutdown_connections 2024-08-15T17:03:04.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/1562894100 wait complete. 2024-08-15T17:03:04.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 Processor -- start 2024-08-15T17:03:04.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- start start 2024-08-15T17:03:04.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 0x7fe4c414cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c4155390 0x7fe4c414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c4167560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4159b80 con 0x7fe4c40686a0 2024-08-15T17:03:04.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4158f70 con 0x7fe4c4155390 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c9ff1700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4159250 con 0x7fe4c4155760 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c8fef700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 0x7fe4c414cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.502+0000 7fe4c8fef700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 0x7fe4c414cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33618/0 (socket says 172.21.15.146:33618) 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c97f0700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c4167560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 -- 172.21.15.146:0/877766696 learned_addr learned my addr 172.21.15.146:0/877766696 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c3fff700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c4155390 0x7fe4c414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 -- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 msgr2=0x7fe4c4167560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.518 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4c4155760 0x7fe4c4167560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 -- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c4155390 msgr2=0x7fe4c414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe4c4155390 0x7fe4c414f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc00abf0 con 0x7fe4c40686a0 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c8fef700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 0x7fe4c414cf30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00c590 tx=0x7fe4ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac011590 con 0x7fe4c40686a0 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac011710 con 0x7fe4c40686a0 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c4159530 con 0x7fe4c40686a0 2024-08-15T17:03:04.519 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac011ad0 con 0x7fe4c40686a0 2024-08-15T17:03:04.520 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c414d470 con 0x7fe4c40686a0 2024-08-15T17:03:04.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe4c40644b0 con 0x7fe4c40686a0 2024-08-15T17:03:04.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.506+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe4ac01b3b0 con 0x7fe4c40686a0 2024-08-15T17:03:04.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4a4060860 0x7fe4a4062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.521 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe4ac07bd00 con 0x7fe4c40686a0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c3fff700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4a4060860 0x7fe4a4062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7fe4a4066750 0x7fe4a4068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe4a4069210 con 0x7fe4a4066750 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ac07e340 con 0x7fe4c40686a0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c97f0700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7fe4a4066750 0x7fe4a4068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c3fff700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4a4060860 0x7fe4a4062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe4c41545c0 tx=0x7fe4b40091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c97f0700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7fe4a4066750 0x7fe4a4068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.522 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.510+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== osd.3 v2:172.21.15.146:6800/24249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe4a4069210 con 0x7fe4a4066750 2024-08-15T17:03:04.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 --> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe4c41515c0 con 0x7fe4a4066750 2024-08-15T17:03:04.537 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c0ff9700 1 -- 172.21.15.146:0/877766696 <== osd.3 v2:172.21.15.146:6800/24249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe4c41515c0 con 0x7fe4a4066750 2024-08-15T17:03:04.537 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:04.538 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:04.538 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:04.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7fe4a4066750 msgr2=0x7fe4a4068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:6800/24249,v1:172.21.15.146:6801/24249] conn(0x7fe4a4066750 0x7fe4a4068b60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4a4060860 msgr2=0x7fe4a4062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4a4060860 0x7fe4a4062d10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe4c41545c0 tx=0x7fe4b40091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.539 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 msgr2=0x7fe4c414cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 --2- 172.21.15.146:0/877766696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe4c40686a0 0x7fe4c414cf30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00c590 tx=0x7fe4ac00c8a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c97f0700 1 -- 172.21.15.146:0/877766696 reap_dead start 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 shutdown_connections 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 >> 172.21.15.146:0/877766696 conn(0x7fe4c40ccfe0 msgr2=0x7fe4c40b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 shutdown_connections 2024-08-15T17:03:04.540 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.526+0000 7fe4c9ff1700 1 -- 172.21.15.146:0/877766696 wait complete. 2024-08-15T17:03:04.550 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-15T17:03:04.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 Processor -- start 2024-08-15T17:03:04.660 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- start start 2024-08-15T17:03:04.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 0x7fc948159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7fc948153e40 0x7fc94806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc948157810 con 0x7fc94815a180 2024-08-15T17:03:04.662 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc948157dd0 con 0x7fc948153e40 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815a700 con 0x7fc948153a70 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94cd37700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc94815a180 0x7fc9480a9e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41888/0 (socket says 172.21.15.146:41888) 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94cd37700 1 -- 172.21.15.146:0/1095579855 learned_addr learned my addr 172.21.15.146:0/1095579855 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc93ffff700 1 --2- 172.21.15.146:0/1095579855 >> v2:172.21.15.146:3300/0 conn(0x7fc948153e40 0x7fc94806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 --2- 172.21.15.146:0/1095579855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 0x7fc948159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc9477fe700 1 -- 172.21.15.146:0/1095579855 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873652675 0 0) 0x7fc948157810 con 0x7fc94815a180 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc9477fe700 1 -- 172.21.15.146:0/1095579855 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc934003570 con 0x7fc94815a180 2024-08-15T17:03:04.663 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 -- 172.21.15.146:0/1095579855 >> v2:172.21.15.146:3300/0 conn(0x7fc948153e40 msgr2=0x7fc94806e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 --2- 172.21.15.146:0/1095579855 >> v2:172.21.15.146:3300/0 conn(0x7fc948153e40 0x7fc94806e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 -- 172.21.15.146:0/1095579855 >> v1:172.21.15.146:6789/0 conn(0x7fc94815a180 legacy=0x7fc9480a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 -- 172.21.15.146:0/1095579855 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94815a9e0 con 0x7fc948153a70 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc947fff700 1 --2- 172.21.15.146:0/1095579855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 0x7fc948159c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098e0 tx=0x7fc9380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636218706cde0e8f server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc9477fe700 1 -- 172.21.15.146:0/1095579855 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011630 con 0x7fc948153a70 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc9477fe700 1 -- 172.21.15.146:0/1095579855 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938011790 con 0x7fc948153a70 2024-08-15T17:03:04.664 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc9477fe700 1 -- 172.21.15.146:0/1095579855 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938011b30 con 0x7fc948153a70 2024-08-15T17:03:04.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- 172.21.15.146:0/1095579855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 msgr2=0x7fc948159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 --2- 172.21.15.146:0/1095579855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 0x7fc948159c40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc9380098e0 tx=0x7fc9380091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- 172.21.15.146:0/1095579855 shutdown_connections 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 --2- 172.21.15.146:0/1095579855 >> v2:172.21.15.146:3300/0 conn(0x7fc948153e40 0x7fc94806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 --2- 172.21.15.146:0/1095579855 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc948153a70 0x7fc948159c40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- 172.21.15.146:0/1095579855 >> 172.21.15.146:0/1095579855 conn(0x7fc948067810 msgr2=0x7fc948067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- 172.21.15.146:0/1095579855 shutdown_connections 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.650+0000 7fc94e79a700 1 -- 172.21.15.146:0/1095579855 wait complete. 2024-08-15T17:03:04.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 Processor -- start 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- start start 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc948153e40 0x7fc94814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc94815a180 0x7fc94815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94814d280 con 0x7fc948153a70 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815d750 con 0x7fc948153e40 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94815cb40 con 0x7fc94815a180 2024-08-15T17:03:04.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33634/0 (socket says 172.21.15.146:33634) 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 -- 172.21.15.146:0/2693016673 learned_addr learned my addr 172.21.15.146:0/2693016673 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94cd37700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc94815a180 0x7fc94815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc93ffff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc948153e40 0x7fc94814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 -- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc94815a180 msgr2=0x7fc94815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc94815a180 0x7fc94815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 -- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc948153e40 msgr2=0x7fc94814c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc948153e40 0x7fc94814c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 -- 172.21.15.146:0/2693016673 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938009420 con 0x7fc948153a70 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc947fff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc938009ed0 tx=0x7fc938017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938000d10 con 0x7fc948153a70 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938011b10 con 0x7fc948153a70 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94815ce20 con 0x7fc948153a70 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938025620 con 0x7fc948153a70 2024-08-15T17:03:04.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.654+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94815d3e0 con 0x7fc948153a70 2024-08-15T17:03:04.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9480645c0 con 0x7fc948153a70 2024-08-15T17:03:04.672 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc938025a10 con 0x7fc948153a70 2024-08-15T17:03:04.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc944ff9700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 0x7fc928062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fc93807dff0 con 0x7fc948153a70 2024-08-15T17:03:04.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc93ffff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 0x7fc928062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.673 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.658+0000 7fc93ffff700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 0x7fc928062d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94815c030 tx=0x7fc9300091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.679 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.666+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc93804e640 con 0x7fc948153a70 2024-08-15T17:03:04.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.822+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc94815d890 con 0x7fc928060890 2024-08-15T17:03:04.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.826+0000 7fc944ff9700 1 -- 172.21.15.146:0/2693016673 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc930011a20 con 0x7fc928060890 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 msgr2=0x7fc928062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 0x7fc928062d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc94815c030 tx=0x7fc9300091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 msgr2=0x7fc94815dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc938009ed0 tx=0x7fc938017000 comp rx=0 tx=0).stop 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 shutdown_connections 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc928060890 0x7fc928062d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc94815a180 0x7fc94815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc948153e40 0x7fc94814c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 --2- 172.21.15.146:0/2693016673 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc948153a70 0x7fc94815dbe0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 >> 172.21.15.146:0/2693016673 conn(0x7fc948067810 msgr2=0x7fc9480ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 shutdown_connections 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.830+0000 7fc94e79a700 1 -- 172.21.15.146:0/2693016673 wait complete. 2024-08-15T17:03:04.841 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 3 to deep-scrub 2024-08-15T17:03:04.856 INFO:tasks.ceph:Scrubbing osd.4 2024-08-15T17:03:04.856 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:04.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.954+0000 7f505e306700 1 Processor -- start 2024-08-15T17:03:04.967 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.954+0000 7f505e306700 1 -- start start 2024-08-15T17:03:04.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505e306700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50580abc20 0x7f50580abff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505e306700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505806ddd0 con 0x7f505806b210 2024-08-15T17:03:04.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505e306700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505806d1c0 con 0x7f505806e260 2024-08-15T17:03:04.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505e306700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505806d4a0 con 0x7f50580abc20 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505db05700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f505806b210 0x7f50580adc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41904/0 (socket says 172.21.15.146:41904) 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505db05700 1 -- 172.21.15.146:0/502628455 learned_addr learned my addr 172.21.15.146:0/502628455 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274107062 0 0) 0x7f505806ddd0 con 0x7f505806b210 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50580abc20 0x7f50580abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5040003570 con 0x7f505806b210 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1748391754 0 0) 0x7f5040003570 con 0x7f505806b210 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50580abc20 msgr2=0x7f50580abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50580abc20 0x7f50580abff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 >> v1:172.21.15.161:6789/0 conn(0x7f505806e260 legacy=0x7f50580aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505806d780 con 0x7f505806b210 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7f50480017f0 con 0x7f505806b210 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 >> v1:172.21.15.146:6789/0 conn(0x7f505806b210 legacy=0x7f50580adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5040003a90 0x7f5040003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040004810 0x7f5040004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.974 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040005210 0x7f504000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505806d4a0 con 0x7f5040003a90 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504000c0e0 con 0x7f5040005210 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504000c2a0 con 0x7f5040004810 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50580abc20 0x7f50580abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5040003a90 0x7f5040003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505db05700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040005210 0x7f504000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505cb03700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040004810 0x7f5040004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.975 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 -- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040004810 msgr2=0x7f5040004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040004810 0x7f5040004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 -- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040005210 msgr2=0x7f504000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5040005210 0x7f504000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 -- 172.21.15.146:0/502628455 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040003570 con 0x7f5040003a90 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505cb03700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040004810 0x7f5040004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f505d304700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5040003a90 0x7f5040003ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb4caf5d021fffd server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000c070 con 0x7f5040003a90 2024-08-15T17:03:04.976 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050013410 con 0x7f5040003a90 2024-08-15T17:03:04.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.958+0000 7f5057fff700 1 -- 172.21.15.146:0/502628455 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500135a0 con 0x7f5040003a90 2024-08-15T17:03:04.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- 172.21.15.146:0/502628455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5040003a90 msgr2=0x7f5040003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 --2- 172.21.15.146:0/502628455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5040003a90 0x7f5040003ea0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050008040 comp rx=0 tx=0).stop 2024-08-15T17:03:04.977 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505db05700 1 -- 172.21.15.146:0/502628455 reap_dead start 2024-08-15T17:03:04.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- 172.21.15.146:0/502628455 shutdown_connections 2024-08-15T17:03:04.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- 172.21.15.146:0/502628455 >> 172.21.15.146:0/502628455 conn(0x7f5058066e00 msgr2=0x7f5058067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:04.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- 172.21.15.146:0/502628455 shutdown_connections 2024-08-15T17:03:04.978 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- 172.21.15.146:0/502628455 wait complete. 2024-08-15T17:03:04.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 Processor -- start 2024-08-15T17:03:04.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.962+0000 7f505e306700 1 -- start start 2024-08-15T17:03:04.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040003a90 0x7f50581337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5058163700 0x7f5058150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.981 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 0x7f505814f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50581323d0 con 0x7f505814ef00 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505814f890 con 0x7f5058163700 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5058133f80 con 0x7f5040003a90 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 0x7f505814f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505cb03700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5058163700 0x7f5058150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.982 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 0x7f505814f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33640/0 (socket says 172.21.15.146:33640) 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505d304700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040003a90 0x7f50581337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 -- 172.21.15.146:0/2335226216 learned_addr learned my addr 172.21.15.146:0/2335226216 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 -- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040003a90 msgr2=0x7f50581337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5040003a90 0x7f50581337a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 -- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5058163700 msgr2=0x7f5058150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5058163700 0x7f5058150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058131ae0 con 0x7f505814ef00 2024-08-15T17:03:04.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505db05700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 0x7f505814f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5040004790 tx=0x7f50480021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800ac90 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504800ae10 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058133ce0 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5048013640 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.966+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058132060 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50580644b0 con 0x7f505814ef00 2024-08-15T17:03:04.984 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f5048013a30 con 0x7f505814ef00 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5044060860 0x7f5044062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f504807d150 con 0x7f505814ef00 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f5044066750 0x7f5044068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5044069210 con 0x7f5044066750 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504807f6b0 con 0x7f505814ef00 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f505d304700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5044060860 0x7f5044062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.985 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f505cb03700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f5044066750 0x7f5044068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:04.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f505d304700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5044060860 0x7f5044062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050019000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.970+0000 7f505cb03700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f5044066750 0x7f5044068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:04.986 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.974+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== osd.4 v2:172.21.15.161:6824/24465 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5044069210 con 0x7f5044066750 2024-08-15T17:03:05.000 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 --> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5058068700 con 0x7f5044066750 2024-08-15T17:03:05.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f50557fa700 1 -- 172.21.15.146:0/2335226216 <== osd.4 v2:172.21.15.161:6824/24465 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5058068700 con 0x7f5044066750 2024-08-15T17:03:05.001 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:05.001 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:05.002 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:05.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f5044066750 msgr2=0x7f5044068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6824/24465,v1:172.21.15.161:6825/24465] conn(0x7f5044066750 0x7f5044068b60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5044060860 msgr2=0x7f5044062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f5044060860 0x7f5044062d10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050019000 comp rx=0 tx=0).stop 2024-08-15T17:03:05.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 msgr2=0x7f505814f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 --2- 172.21.15.146:0/2335226216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f505814ef00 0x7f505814f350 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5040004790 tx=0x7f50480021d0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505db05700 1 -- 172.21.15.146:0/2335226216 reap_dead start 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 shutdown_connections 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 >> 172.21.15.146:0/2335226216 conn(0x7f5058066e00 msgr2=0x7f505806a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 shutdown_connections 2024-08-15T17:03:05.004 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:04.990+0000 7f505e306700 1 -- 172.21.15.146:0/2335226216 wait complete. 2024-08-15T17:03:05.014 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-15T17:03:05.131 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.118+0000 7f0374568700 1 Processor -- start 2024-08-15T17:03:05.132 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 -- start start 2024-08-15T17:03:05.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c15a620 0x7f036c0ab170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.133 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f036c0b0f10 0x7f036c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0ae530 con 0x7f036c0ae0d0 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0aff10 con 0x7f036c0b0f10 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0374568700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0a9160 con 0x7f036c15a620 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f036c0b0f10 0x7f036c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f036c0b0f10 0x7f036c0adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33656/0 (socket says 172.21.15.146:33656) 2024-08-15T17:03:05.134 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 -- 172.21.15.146:0/3617136187 learned_addr learned my addr 172.21.15.146:0/3617136187 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0372b05700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f036c0ae0d0 0x7f036c06e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41906/0 (socket says 172.21.15.146:41906) 2024-08-15T17:03:05.135 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0372304700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c15a620 0x7f036c0ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 -- 172.21.15.146:0/3617136187 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c15a620 msgr2=0x7f036c0ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c15a620 0x7f036c0ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.136 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 -- 172.21.15.146:0/3617136187 >> v1:172.21.15.146:6789/0 conn(0x7f036c0ae0d0 legacy=0x7f036c06e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 -- 172.21.15.146:0/3617136187 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0b1680 con 0x7f036c0b0f10 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371b03700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c0b0f10 0x7f036c0adb90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f035c0099b0 tx=0x7f035c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e819c41c46d0c44e server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371302700 1 -- 172.21.15.146:0/3617136187 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c009ea0 con 0x7f036c0b0f10 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371302700 1 -- 172.21.15.146:0/3617136187 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c00b860 con 0x7f036c0b0f10 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.122+0000 7f0371302700 1 -- 172.21.15.146:0/3617136187 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c00bc00 con 0x7f036c0b0f10 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- 172.21.15.146:0/3617136187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c0b0f10 msgr2=0x7f036c0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.137 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c0b0f10 0x7f036c0adb90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f035c0099b0 tx=0x7f035c009250 comp rx=0 tx=0).stop 2024-08-15T17:03:05.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- 172.21.15.146:0/3617136187 shutdown_connections 2024-08-15T17:03:05.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c0b0f10 0x7f036c0adb90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- 172.21.15.146:0/3617136187 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c15a620 0x7f036c0ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- 172.21.15.146:0/3617136187 >> 172.21.15.146:0/3617136187 conn(0x7f036c067810 msgr2=0x7f036c067c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.138 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- 172.21.15.146:0/3617136187 shutdown_connections 2024-08-15T17:03:05.139 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- 172.21.15.146:0/3617136187 wait complete. 2024-08-15T17:03:05.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 Processor -- start 2024-08-15T17:03:05.141 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- start start 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f036c0ae0d0 0x7f036c14bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c0b0f10 0x7f036c0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c9cd0 con 0x7f036c15a620 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c6230 con 0x7f036c0ae0d0 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0374568700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0c67b0 con 0x7f036c0b0f10 2024-08-15T17:03:05.142 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0372b05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0372b05700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33666/0 (socket says 172.21.15.146:33666) 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0372b05700 1 -- 172.21.15.146:0/1481313642 learned_addr learned my addr 172.21.15.146:0/1481313642 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0371b03700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c0b0f10 0x7f036c0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.126+0000 7f0372304700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f036c0ae0d0 0x7f036c14bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 -- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c0b0f10 msgr2=0x7f036c0c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c0b0f10 0x7f036c0c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.143 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 -- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f036c0ae0d0 msgr2=0x7f036c14bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f036c0ae0d0 0x7f036c14bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 -- 172.21.15.146:0/1481313642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0094f0 con 0x7f036c15a620 2024-08-15T17:03:05.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0372b05700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f036400cc70 tx=0x7f0364001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.144 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364004bb0 con 0x7f036c15a620 2024-08-15T17:03:05.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0364004d10 con 0x7f036c15a620 2024-08-15T17:03:05.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c169b20 con 0x7f036c15a620 2024-08-15T17:03:05.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400ab90 con 0x7f036c15a620 2024-08-15T17:03:05.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c0caf30 con 0x7f036c15a620 2024-08-15T17:03:05.146 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c0645c0 con 0x7f036c15a620 2024-08-15T17:03:05.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f03640070f0 con 0x7f036c15a620 2024-08-15T17:03:05.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.130+0000 7f03627fc700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 0x7f0350062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.147 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.134+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f036407c400 con 0x7f036c15a620 2024-08-15T17:03:05.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.134+0000 7f0372304700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 0x7f0350062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.148 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.134+0000 7f0372304700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 0x7f0350062d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0368009d40 tx=0x7f0368008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.154 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.142+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f036404cc30 con 0x7f036c15a620 2024-08-15T17:03:05.323 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f036c0ae530 con 0x7f0350060890 2024-08-15T17:03:05.324 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f03627fc700 1 -- 172.21.15.146:0/1481313642 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f036c0ae530 con 0x7f0350060890 2024-08-15T17:03:05.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 msgr2=0x7f0350062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 0x7f0350062d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0368009d40 tx=0x7f0368008040 comp rx=0 tx=0).stop 2024-08-15T17:03:05.327 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 msgr2=0x7f036c0c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f036400cc70 tx=0x7f0364001a70 comp rx=0 tx=0).stop 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 shutdown_connections 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f0350060890 0x7f0350062d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f036c15a620 0x7f036c0c84f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f036c0b0f10 0x7f036c0c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 --2- 172.21.15.146:0/1481313642 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f036c0ae0d0 0x7f036c14bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 >> 172.21.15.146:0/1481313642 conn(0x7f036c067810 msgr2=0x7f036c06a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.328 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 shutdown_connections 2024-08-15T17:03:05.329 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.314+0000 7f0374568700 1 -- 172.21.15.146:0/1481313642 wait complete. 2024-08-15T17:03:05.329 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 4 to deep-scrub 2024-08-15T17:03:05.338 INFO:tasks.ceph:Scrubbing osd.5 2024-08-15T17:03:05.339 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:05.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.442+0000 7f9ba061f700 1 Processor -- start 2024-08-15T17:03:05.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.442+0000 7f9ba061f700 1 -- start start 2024-08-15T17:03:05.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b98157740 0x7f9b980c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9b98158220 0x7f9b98160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980c4510 con 0x7f9b980c8c10 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b980c49a0 con 0x7f9b98157740 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b980c3900 con 0x7f9b98158220 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b98157740 0x7f9b980c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f9b98157740 0x7f9b980c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33674/0 (socket says 172.21.15.146:33674) 2024-08-15T17:03:05.457 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 -- 172.21.15.146:0/2935952394 learned_addr learned my addr 172.21.15.146:0/2935952394 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.458 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9ebbc700 1 --2- 172.21.15.146:0/2935952394 >> v2:172.21.15.161:3300/0 conn(0x7f9b98158220 0x7f9b98160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9d3b9700 1 -- 172.21.15.146:0/2935952394 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628362844 0 0) 0x7f9b980c4510 con 0x7f9b980c8c10 2024-08-15T17:03:05.459 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 -- 172.21.15.146:0/2935952394 >> v2:172.21.15.161:3300/0 conn(0x7f9b98158220 msgr2=0x7f9b98160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 --2- 172.21.15.146:0/2935952394 >> v2:172.21.15.161:3300/0 conn(0x7f9b98158220 0x7f9b98160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 -- 172.21.15.146:0/2935952394 >> v1:172.21.15.161:6789/0 conn(0x7f9b980c8c10 legacy=0x7f9b98157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 -- 172.21.15.146:0/2935952394 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980c3be0 con 0x7f9b98157740 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9e3bb700 1 --2- 172.21.15.146:0/2935952394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980c86d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9b980caa40 tx=0x7f9b94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399de75c7509f4a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9d3b9700 1 -- 172.21.15.146:0/2935952394 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94009e70 con 0x7f9b98157740 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9d3b9700 1 -- 172.21.15.146:0/2935952394 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9400b860 con 0x7f9b98157740 2024-08-15T17:03:05.460 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9b9d3b9700 1 -- 172.21.15.146:0/2935952394 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400bc00 con 0x7f9b98157740 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 -- 172.21.15.146:0/2935952394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 msgr2=0x7f9b980c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 --2- 172.21.15.146:0/2935952394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980c86d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9b980caa40 tx=0x7f9b94009250 comp rx=0 tx=0).stop 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 -- 172.21.15.146:0/2935952394 shutdown_connections 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.446+0000 7f9ba061f700 1 --2- 172.21.15.146:0/2935952394 >> v2:172.21.15.161:3300/0 conn(0x7f9b98158220 0x7f9b98160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 --2- 172.21.15.146:0/2935952394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980c86d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- 172.21.15.146:0/2935952394 >> 172.21.15.146:0/2935952394 conn(0x7f9b980cab00 msgr2=0x7f9b980caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- 172.21.15.146:0/2935952394 shutdown_connections 2024-08-15T17:03:05.461 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- 172.21.15.146:0/2935952394 wait complete. 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 Processor -- start 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- start start 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980c8c10 0x7f9b980b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98158220 0x7f9b98152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98068b00 con 0x7f9b98157740 2024-08-15T17:03:05.464 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b2e50 con 0x7f9b98158220 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b3130 con 0x7f9b980c8c10 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9e3bb700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980c8c10 0x7f9b980b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33682/0 (socket says 172.21.15.146:33682) 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 -- 172.21.15.146:0/228574928 learned_addr learned my addr 172.21.15.146:0/228574928 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9ebbc700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98158220 0x7f9b98152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 -- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980c8c10 msgr2=0x7f9b980b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.465 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980c8c10 0x7f9b980b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 -- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98158220 msgr2=0x7f9b98152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b98158220 0x7f9b98152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940094f0 con 0x7f9b98157740 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b9dbba700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980a98d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c7e0 tx=0x7f9b8c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011590 con 0x7f9b98157740 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c011710 con 0x7f9b98157740 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b980b3410 con 0x7f9b98157740 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011ad0 con 0x7f9b98157740 2024-08-15T17:03:05.466 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.450+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b980a9e10 con 0x7f9b98157740 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b980638f0 con 0x7f9b98157740 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9b8c01b3b0 con 0x7f9b98157740 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9b7c060860 0x7f9b7c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f9b8c07beb0 con 0x7f9b98157740 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7f9b7c066750 0x7f9b7c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b7c069210 con 0x7f9b7c066750 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8c07e410 con 0x7f9b98157740 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.454+0000 7f9b9e3bb700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9b7c060860 0x7f9b7c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.458+0000 7f9b9ebbc700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7f9b7c066750 0x7f9b7c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.458+0000 7f9b9e3bb700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9b7c060860 0x7f9b7c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9b980caa40 tx=0x7f9b94017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.458+0000 7f9b9ebbc700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7f9b7c066750 0x7f9b7c068b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.458+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== osd.5 v2:172.21.15.161:6816/24464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f9b7c069210 con 0x7f9b7c066750 2024-08-15T17:03:05.489 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 --> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9b980b3f00 con 0x7f9b7c066750 2024-08-15T17:03:05.490 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9b8a7fc700 1 -- 172.21.15.146:0/228574928 <== osd.5 v2:172.21.15.161:6816/24464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9b980b3f00 con 0x7f9b7c066750 2024-08-15T17:03:05.490 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:05.490 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:05.491 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7f9b7c066750 msgr2=0x7f9b7c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6816/24464,v1:172.21.15.161:6817/24464] conn(0x7f9b7c066750 0x7f9b7c068b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9b7c060860 msgr2=0x7f9b7c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9b7c060860 0x7f9b7c062d10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9b980caa40 tx=0x7f9b94017000 comp rx=0 tx=0).stop 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 msgr2=0x7f9b980a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.492 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 --2- 172.21.15.146:0/228574928 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9b98157740 0x7f9b980a98d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c7e0 tx=0x7f9b8c007d90 comp rx=0 tx=0).stop 2024-08-15T17:03:05.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9b9ebbc700 1 -- 172.21.15.146:0/228574928 reap_dead start 2024-08-15T17:03:05.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 shutdown_connections 2024-08-15T17:03:05.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 >> 172.21.15.146:0/228574928 conn(0x7f9b980cab00 msgr2=0x7f9b9806a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 shutdown_connections 2024-08-15T17:03:05.493 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.478+0000 7f9ba061f700 1 -- 172.21.15.146:0/228574928 wait complete. 2024-08-15T17:03:05.502 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-15T17:03:05.618 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.606+0000 7fe5119ef700 1 Processor -- start 2024-08-15T17:03:05.619 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.606+0000 7fe5119ef700 1 -- start start 2024-08-15T17:03:05.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe50c06df10 0x7fe50c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c0af2b0 con 0x7fe50c155d20 2024-08-15T17:03:05.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50c0b07a0 con 0x7fe50c06df10 2024-08-15T17:03:05.620 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c06e8d0 con 0x7fe50c0b0350 2024-08-15T17:03:05.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe50c06df10 0x7fe50c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:54044/0 (socket says 172.21.15.146:54044) 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe50c06df10 0x7fe50c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.146:43112/0 (socket says 172.21.15.146:43112) 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50bfff700 1 -- 172.21.15.146:0/2882712829 learned_addr learned my addr 172.21.15.146:0/2882712829 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5109ed700 1 --1- 172.21.15.146:0/2882712829 >> v1:172.21.15.161:6789/0 conn(0x7fe50c155d20 0x7fe50c0aea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:58342/0 (socket says 172.21.15.146:58342) 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 -- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 msgr2=0x7fe50c0ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 -- 172.21.15.146:0/2882712829 >> v1:172.21.15.161:6789/0 conn(0x7fe50c155d20 legacy=0x7fe50c0aea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.623 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 -- 172.21.15.146:0/2882712829 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0af590 con 0x7fe50c06df10 2024-08-15T17:03:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50bfff700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5111ee700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c06df10 0x7fe50c06e390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe504009e90 tx=0x7fe504009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6352b5d488f5fb0d server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50b7fe700 1 -- 172.21.15.146:0/2882712829 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400e070 con 0x7fe50c06df10 2024-08-15T17:03:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50b7fe700 1 -- 172.21.15.146:0/2882712829 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50400bbf0 con 0x7fe50c06df10 2024-08-15T17:03:05.624 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe50b7fe700 1 -- 172.21.15.146:0/2882712829 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504012430 con 0x7fe50c06df10 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c06df10 msgr2=0x7fe50c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c06df10 0x7fe50c06e390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe504009e90 tx=0x7fe504009780 comp rx=0 tx=0).stop 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- 172.21.15.146:0/2882712829 shutdown_connections 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c06df10 0x7fe50c06e390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 --2- 172.21.15.146:0/2882712829 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c0b0350 0x7fe50c0ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.626 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- 172.21.15.146:0/2882712829 >> 172.21.15.146:0/2882712829 conn(0x7fe50c0678d0 msgr2=0x7fe50c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- 172.21.15.146:0/2882712829 shutdown_connections 2024-08-15T17:03:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.610+0000 7fe5119ef700 1 -- 172.21.15.146:0/2882712829 wait complete. 2024-08-15T17:03:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 Processor -- start 2024-08-15T17:03:05.627 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 -- start start 2024-08-15T17:03:05.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c06df10 0x7fe50c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.629 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c155d20 0x7fe50c13ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c151c50 con 0x7fe50c0b0350 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c151f30 con 0x7fe50c155d20 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5119ef700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c152210 con 0x7fe50c06df10 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.630 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33698/0 (socket says 172.21.15.146:33698) 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe50bfff700 1 -- 172.21.15.146:0/802785685 learned_addr learned my addr 172.21.15.146:0/802785685 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5109ed700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c06df10 0x7fe50c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.614+0000 7fe5111ee700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c155d20 0x7fe50c13ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 -- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c06df10 msgr2=0x7fe50c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c06df10 0x7fe50c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 -- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c155d20 msgr2=0x7fe50c13ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.631 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c155d20 0x7fe50c13ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 -- 172.21.15.146:0/802785685 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc0094f0 con 0x7fe50c0b0350 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe50bfff700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ec50 tx=0x7fe4fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00c850 con 0x7fe50c0b0350 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc00c9b0 con 0x7fe50c0b0350 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe504009a20 con 0x7fe50c0b0350 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00cd50 con 0x7fe50c0b0350 2024-08-15T17:03:05.632 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c151860 con 0x7fe50c0b0350 2024-08-15T17:03:05.633 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fe4fc013680 con 0x7fe50c0b0350 2024-08-15T17:03:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.618+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c0643a0 con 0x7fe50c0b0350 2024-08-15T17:03:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.622+0000 7fe508ff9700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 0x7fe4ec062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.622+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe4fc07cbc0 con 0x7fe50c0b0350 2024-08-15T17:03:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.622+0000 7fe5109ed700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 0x7fe4ec062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.634 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.622+0000 7fe5109ed700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 0x7fe4ec062d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004fd0 tx=0x7fe4f4007eb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.641 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.630+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fe4fc04d2b0 con 0x7fe50c0b0350 2024-08-15T17:03:05.811 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.798+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe50c06f5f0 con 0x7fe4ec060890 2024-08-15T17:03:05.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe508ff9700 1 -- 172.21.15.146:0/802785685 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe50c06f5f0 con 0x7fe4ec060890 2024-08-15T17:03:05.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 msgr2=0x7fe4ec062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 0x7fe4ec062d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4004fd0 tx=0x7fe4f4007eb0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.815 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 msgr2=0x7fe50c13a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00ec50 tx=0x7fe4fc009250 comp rx=0 tx=0).stop 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 shutdown_connections 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fe4ec060890 0x7fe4ec062d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe50c155d20 0x7fe50c13ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe50c0b0350 0x7fe50c13a6e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 --2- 172.21.15.146:0/802785685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe50c06df10 0x7fe50c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.816 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 >> 172.21.15.146:0/802785685 conn(0x7fe50c0678d0 msgr2=0x7fe50c0ab9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 shutdown_connections 2024-08-15T17:03:05.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.802+0000 7fe5119ef700 1 -- 172.21.15.146:0/802785685 wait complete. 2024-08-15T17:03:05.817 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 5 to deep-scrub 2024-08-15T17:03:05.827 INFO:tasks.ceph:Scrubbing osd.6 2024-08-15T17:03:05.827 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:05.938 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.926+0000 7fb319a07700 1 Processor -- start 2024-08-15T17:03:05.939 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.926+0000 7fb319a07700 1 -- start start 2024-08-15T17:03:05.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319a07700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb3140ce640 0x7fb3140b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319a07700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140c5bb0 con 0x7fb3140cea10 2024-08-15T17:03:05.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319a07700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140c4fa0 con 0x7fb3140cc400 2024-08-15T17:03:05.944 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319a07700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140c5280 con 0x7fb3140ce640 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fb3140cea10 0x7fb3140cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41918/0 (socket says 172.21.15.146:41918) 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 -- 172.21.15.146:0/1231259584 learned_addr learned my addr 172.21.15.146:0/1231259584 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb318a05700 1 --2- 172.21.15.146:0/1231259584 >> v2:172.21.15.161:3300/0 conn(0x7fb3140ce640 0x7fb3140b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534870044 0 0) 0x7fb3140c5bb0 con 0x7fb3140cea10 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb304003570 con 0x7fb3140cea10 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082904075 0 0) 0x7fb3140c4fa0 con 0x7fb3140cc400 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3140c5bb0 con 0x7fb3140cc400 2024-08-15T17:03:05.945 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633881371 0 0) 0x7fb304003570 con 0x7fb3140cea10 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 >> v2:172.21.15.161:3300/0 conn(0x7fb3140ce640 msgr2=0x7fb3140b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 --2- 172.21.15.146:0/1231259584 >> v2:172.21.15.161:3300/0 conn(0x7fb3140ce640 0x7fb3140b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 >> v1:172.21.15.161:6789/0 conn(0x7fb3140cc400 legacy=0x7fb3140c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140c5560 con 0x7fb3140cea10 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3985598646 0 0) 0x7fb308001470 con 0x7fb3140cea10 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 >> v1:172.21.15.146:6789/0 conn(0x7fb3140cea10 legacy=0x7fb3140cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb304003bf0 0x7fb304003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.946 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 0x7fb304004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3040051c0 0x7fb30400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c5280 con 0x7fb304004810 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30400c160 con 0x7fb304003bf0 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30400c320 con 0x7fb3040051c0 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 0x7fb304004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb318a05700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb304003bf0 0x7fb304003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319206700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3040051c0 0x7fb30400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.947 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 -- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3040051c0 msgr2=0x7fb30400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3040051c0 0x7fb30400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 -- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb304003bf0 msgr2=0x7fb304003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:05.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb304003bf0 0x7fb304003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 -- 172.21.15.146:0/1231259584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb304003570 con 0x7fb304004810 2024-08-15T17:03:05.948 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319206700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb3040051c0 0x7fb30400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb319206700 1 -- 172.21.15.146:0/1231259584 reap_dead start 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb313fff700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 0x7fb304004c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c2a0 tx=0x7fb308001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32128fbcf8ebfa4b server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30800cb70 con 0x7fb304004810 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30800ccd0 con 0x7fb304004810 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.930+0000 7fb3137fe700 1 -- 172.21.15.146:0/1231259584 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb308005b70 con 0x7fb304004810 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 msgr2=0x7fb304004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.949 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 0x7fb304004c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb30800c2a0 tx=0x7fb308001e10 comp rx=0 tx=0).stop 2024-08-15T17:03:05.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- 172.21.15.146:0/1231259584 shutdown_connections 2024-08-15T17:03:05.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 --2- 172.21.15.146:0/1231259584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb304004810 0x7fb304004c80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- 172.21.15.146:0/1231259584 >> 172.21.15.146:0/1231259584 conn(0x7fb314099410 msgr2=0x7fb3140675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- 172.21.15.146:0/1231259584 shutdown_connections 2024-08-15T17:03:05.950 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- 172.21.15.146:0/1231259584 wait complete. 2024-08-15T17:03:05.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 Processor -- start 2024-08-15T17:03:05.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.934+0000 7fb319a07700 1 -- start start 2024-08-15T17:03:05.954 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb304004810 0x7fb314151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3140b4f40 0x7fb3140c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 0x7fb31415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415e340 con 0x7fb31415d960 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3141528b0 con 0x7fb3140b4f40 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c8470 con 0x7fb304004810 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 0x7fb31415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 0x7fb31415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33730/0 (socket says 172.21.15.146:33730) 2024-08-15T17:03:05.955 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 -- 172.21.15.146:0/301529571 learned_addr learned my addr 172.21.15.146:0/301529571 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb313fff700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3140b4f40 0x7fb3140c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb318a05700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb304004810 0x7fb314151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 -- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb304004810 msgr2=0x7fb314151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb304004810 0x7fb314151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 -- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3140b4f40 msgr2=0x7fb3140c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb3140b4f40 0x7fb3140c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140c8750 con 0x7fb31415d960 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319206700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 0x7fb31415de00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3140898e0 tx=0x7fb30c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.956 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0032e0 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c003440 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3140c8a30 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c006df0 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3140c8ff0 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.938+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3140c69e0 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fb30c0018f0 con 0x7fb31415d960 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb2f8060890 0x7fb2f8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.957 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fb30c07c350 con 0x7fb31415d960 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7fb2f8066780 0x7fb2f8068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2f8069240 con 0x7fb2f8066780 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb30c07c720 con 0x7fb31415d960 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb318a05700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb2f8060890 0x7fb2f8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb313fff700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7fb2f8066780 0x7fb2f8068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb313fff700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7fb2f8066780 0x7fb2f8068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.958 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.942+0000 7fb318a05700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb2f8060890 0x7fb2f8062d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb300006f00 tx=0x7fb3000091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:05.959 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.946+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== osd.6 v2:172.21.15.161:6800/24462 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fb2f8069240 con 0x7fb2f8066780 2024-08-15T17:03:05.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.958+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 --> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb3140644b0 con 0x7fb2f8066780 2024-08-15T17:03:05.970 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.958+0000 7fb310ff9700 1 -- 172.21.15.146:0/301529571 <== osd.6 v2:172.21.15.161:6800/24462 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb3140644b0 con 0x7fb2f8066780 2024-08-15T17:03:05.971 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:05.971 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:05.971 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:05.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.958+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7fb2f8066780 msgr2=0x7fb2f8068b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6800/24462,v1:172.21.15.161:6801/24462] conn(0x7fb2f8066780 0x7fb2f8068b90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb2f8060890 msgr2=0x7fb2f8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.972 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fb2f8060890 0x7fb2f8062d40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb300006f00 tx=0x7fb3000091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 msgr2=0x7fb31415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 --2- 172.21.15.146:0/301529571 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb31415d960 0x7fb31415de00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb3140898e0 tx=0x7fb30c009040 comp rx=0 tx=0).stop 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319206700 1 -- 172.21.15.146:0/301529571 reap_dead start 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 shutdown_connections 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 >> 172.21.15.146:0/301529571 conn(0x7fb314099410 msgr2=0x7fb3140cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 shutdown_connections 2024-08-15T17:03:05.973 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:05.962+0000 7fb319a07700 1 -- 172.21.15.146:0/301529571 wait complete. 2024-08-15T17:03:05.990 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-15T17:03:06.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 Processor -- start 2024-08-15T17:03:06.104 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 -- start start 2024-08-15T17:03:06.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc60c0b14a0 0x7fc60c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.105 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0cbaa0 con 0x7fc60c0cb390 2024-08-15T17:03:06.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c157630 con 0x7fc60c154900 2024-08-15T17:03:06.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611978700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60c068790 con 0x7fc60c0b14a0 2024-08-15T17:03:06.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611177700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7fc60c0cb390 0x7fc60c160ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41924/0 (socket says 172.21.15.146:41924) 2024-08-15T17:03:06.106 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc611177700 1 -- 172.21.15.146:0/223557317 learned_addr learned my addr 172.21.15.146:0/223557317 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:06.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60bfff700 1 --2- 172.21.15.146:0/223557317 >> v2:172.21.15.161:3300/0 conn(0x7fc60c0b14a0 0x7fc60c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.107 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572057196 0 0) 0x7fc60c0cbaa0 con 0x7fc60c0cb390 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5ec003570 con 0x7fc60c0cb390 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60bfff700 1 -- 172.21.15.146:0/223557317 >> v1:172.21.15.161:6789/0 conn(0x7fc60c154900 legacy=0x7fc60c0b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3847463518 0 0) 0x7fc5ec003570 con 0x7fc60c0cb390 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60bfff700 1 -- 172.21.15.146:0/223557317 >> v1:172.21.15.146:6789/0 conn(0x7fc60c0cb390 legacy=0x7fc60c160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60bfff700 1 -- 172.21.15.146:0/223557317 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c157910 con 0x7fc60c0b14a0 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60bfff700 1 --2- 172.21.15.146:0/223557317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0cae50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc604009e60 tx=0x7fc604009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83aff69c9c11b15c server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.108 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.094+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60400e070 con 0x7fc60c0b14a0 2024-08-15T17:03:06.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60400b850 con 0x7fc60c0b14a0 2024-08-15T17:03:06.109 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc60b7fe700 1 -- 172.21.15.146:0/223557317 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc001470 con 0x7fc60c0b14a0 2024-08-15T17:03:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- 172.21.15.146:0/223557317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 msgr2=0x7fc60c0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 --2- 172.21.15.146:0/223557317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0cae50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc604009e60 tx=0x7fc604009780 comp rx=0 tx=0).stop 2024-08-15T17:03:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- 172.21.15.146:0/223557317 shutdown_connections 2024-08-15T17:03:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 --2- 172.21.15.146:0/223557317 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0cae50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.110 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- 172.21.15.146:0/223557317 >> 172.21.15.146:0/223557317 conn(0x7fc60c0678d0 msgr2=0x7fc60c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- 172.21.15.146:0/223557317 shutdown_connections 2024-08-15T17:03:06.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- 172.21.15.146:0/223557317 wait complete. 2024-08-15T17:03:06.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 Processor -- start 2024-08-15T17:03:06.111 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- start start 2024-08-15T17:03:06.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc60c0cb390 0x7fc60c069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.113 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c147840 con 0x7fc60c154900 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c146c30 con 0x7fc60c0b14a0 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611978700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c146f10 con 0x7fc60c0cb390 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611177700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611177700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33738/0 (socket says 172.21.15.146:33738) 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc611177700 1 -- 172.21.15.146:0/2114818192 learned_addr learned my addr 172.21.15.146:0/2114818192 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.098+0000 7fc60bfff700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc60c0cb390 0x7fc60c069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.114 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc610976700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 -- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc60c0cb390 msgr2=0x7fc60c069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc60c0cb390 0x7fc60c069f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 -- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 msgr2=0x7fc60c0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 -- 172.21.15.146:0/2114818192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc604009a20 con 0x7fc60c154900 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc610976700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611177700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00cb90 tx=0x7fc5fc001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.115 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc001d80 con 0x7fc60c154900 2024-08-15T17:03:06.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc007360 con 0x7fc60c154900 2024-08-15T17:03:06.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc004830 con 0x7fc60c154900 2024-08-15T17:03:06.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c1471f0 con 0x7fc60c154900 2024-08-15T17:03:06.116 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c0c77f0 con 0x7fc60c154900 2024-08-15T17:03:06.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.102+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c1479b0 con 0x7fc60c154900 2024-08-15T17:03:06.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.106+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7fc5fc004ba0 con 0x7fc60c154900 2024-08-15T17:03:06.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.106+0000 7fc608ff9700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 0x7fc5f0062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.106+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fc5fc07bc70 con 0x7fc60c154900 2024-08-15T17:03:06.117 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.106+0000 7fc610976700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 0x7fc5f0062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.118 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.106+0000 7fc610976700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 0x7fc5f0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007fd0 tx=0x7fc5f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.124 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.114+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7fc5fc04c3e0 con 0x7fc60c154900 2024-08-15T17:03:06.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.270+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc60c0643a0 con 0x7fc5f0060860 2024-08-15T17:03:06.283 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc608ff9700 1 -- 172.21.15.146:0/2114818192 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc60c0643a0 con 0x7fc5f0060860 2024-08-15T17:03:06.286 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 msgr2=0x7fc5f0062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 0x7fc5f0062d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc5f4007fd0 tx=0x7fc5f4009040 comp rx=0 tx=0).stop 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 msgr2=0x7fc60c06a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00cb90 tx=0x7fc5fc001a10 comp rx=0 tx=0).stop 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 shutdown_connections 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7fc5f0060860 0x7fc5f0062d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc60c154900 0x7fc60c06a450 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc60c0cb390 0x7fc60c069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.287 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 --2- 172.21.15.146:0/2114818192 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc60c0b14a0 0x7fc60c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 >> 172.21.15.146:0/2114818192 conn(0x7fc60c0678d0 msgr2=0x7fc60c06e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 shutdown_connections 2024-08-15T17:03:06.288 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.274+0000 7fc611978700 1 -- 172.21.15.146:0/2114818192 wait complete. 2024-08-15T17:03:06.288 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 6 to deep-scrub 2024-08-15T17:03:06.298 INFO:tasks.ceph:Scrubbing osd.7 2024-08-15T17:03:06.298 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-15T17:03:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.406+0000 7f393544c700 1 Processor -- start 2024-08-15T17:03:06.418 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.406+0000 7f393544c700 1 -- start start 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39301427c0 0x7f3930068cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3930142b90 0x7f39300b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393006a010 con 0x7f393015cdd0 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- --> v2:172.21.15.146:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300691f0 con 0x7f3930142b90 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3930069490 con 0x7f39301427c0 2024-08-15T17:03:06.421 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3930142b90 0x7f39300b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 --2- >> v2:172.21.15.146:3300/0 conn(0x7f3930142b90 0x7f39300b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33744/0 (socket says 172.21.15.146:33744) 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 -- 172.21.15.146:0/1002374215 learned_addr learned my addr 172.21.15.146:0/1002374215 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392ffff700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39301427c0 0x7f3930068cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 -- 172.21.15.146:0/1002374215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39301427c0 msgr2=0x7f3930068cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39301427c0 0x7f3930068cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 -- 172.21.15.146:0/1002374215 >> v1:172.21.15.161:6789/0 conn(0x7f393015cdd0 legacy=0x7f39300b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.422 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 -- 172.21.15.146:0/1002374215 --> v2:172.21.15.146:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3930069770 con 0x7f3930142b90 2024-08-15T17:03:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392f7fe700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3930142b90 0x7f39300b0d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f39200099b0 tx=0x7f3920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6742590cccc0d546 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392effd700 1 -- 172.21.15.146:0/1002374215 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392000b860 con 0x7f3930142b90 2024-08-15T17:03:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392effd700 1 -- 172.21.15.146:0/1002374215 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392000b9c0 con 0x7f3930142b90 2024-08-15T17:03:06.423 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f392effd700 1 -- 172.21.15.146:0/1002374215 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392000bd60 con 0x7f3930142b90 2024-08-15T17:03:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- 172.21.15.146:0/1002374215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3930142b90 msgr2=0x7f39300b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.424 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3930142b90 0x7f39300b0d70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f39200099b0 tx=0x7f3920009250 comp rx=0 tx=0).stop 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- 172.21.15.146:0/1002374215 shutdown_connections 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3930142b90 0x7f39300b0d70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 --2- 172.21.15.146:0/1002374215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39301427c0 0x7f3930068cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- 172.21.15.146:0/1002374215 >> 172.21.15.146:0/1002374215 conn(0x7f39300cdea0 msgr2=0x7f39300ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- 172.21.15.146:0/1002374215 shutdown_connections 2024-08-15T17:03:06.425 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.410+0000 7f393544c700 1 -- 172.21.15.146:0/1002374215 wait complete. 2024-08-15T17:03:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 Processor -- start 2024-08-15T17:03:06.426 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- start start 2024-08-15T17:03:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 0x7f39300cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3930142b90 0x7f39301696d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f393015cdd0 0x7f39300cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930169c10 con 0x7f39301427c0 2024-08-15T17:03:06.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cb110 con 0x7f3930142b90 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393016aa30 con 0x7f393015cdd0 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 0x7f39300cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 0x7f39300cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33760/0 (socket says 172.21.15.146:33760) 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392f7fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3930142b90 0x7f39301696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 -- 172.21.15.146:0/277058222 learned_addr learned my addr 172.21.15.146:0/277058222 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f3934c4b700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f393015cdd0 0x7f39300cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 -- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f393015cdd0 msgr2=0x7f39300cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f393015cdd0 0x7f39300cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 -- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3930142b90 msgr2=0x7f39301696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3930142b90 0x7f39301696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200094f0 con 0x7f39301427c0 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f392ffff700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 0x7f39300cb4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f392400d8c0 tx=0x7f3924008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39240125f0 con 0x7f39301427c0 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3924012770 con 0x7f39301427c0 2024-08-15T17:03:06.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924012b30 con 0x7f39301427c0 2024-08-15T17:03:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393016a6a0 con 0x7f39301427c0 2024-08-15T17:03:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.414+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3930169e80 con 0x7f39301427c0 2024-08-15T17:03:06.431 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39300644b0 con 0x7f39301427c0 2024-08-15T17:03:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f392401e460 con 0x7f39301427c0 2024-08-15T17:03:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f391c060860 0x7f391c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.432 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f392407d940 con 0x7f39301427c0 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7f391c066750 0x7f391c068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f391c069210 con 0x7f391c066750 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f392407dc30 con 0x7f39301427c0 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f392f7fe700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f391c060860 0x7f391c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.418+0000 7f3934c4b700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7f391c066750 0x7f391c068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.422+0000 7f392f7fe700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f391c060860 0x7f391c062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f39200099b0 tx=0x7f3920017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.422+0000 7f3934c4b700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7f391c066750 0x7f391c068b60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.434 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.422+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== osd.7 v2:172.21.15.161:6808/24463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f391c069210 con 0x7f391c066750 2024-08-15T17:03:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 --> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3930068580 con 0x7f391c066750 2024-08-15T17:03:06.452 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f3917fff700 1 -- 172.21.15.146:0/277058222 <== osd.7 v2:172.21.15.161:6808/24463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3930068580 con 0x7f391c066750 2024-08-15T17:03:06.453 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-08-15T17:03:06.453 INFO:teuthology.orchestra.run.smithi146.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-15T17:03:06.453 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-08-15T17:03:06.454 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7f391c066750 msgr2=0x7f391c068b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6808/24463,v1:172.21.15.161:6809/24463] conn(0x7f391c066750 0x7f391c068b60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f391c060860 msgr2=0x7f391c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f391c060860 0x7f391c062d10 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f39200099b0 tx=0x7f3920017000 comp rx=0 tx=0).stop 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 msgr2=0x7f39300cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 --2- 172.21.15.146:0/277058222 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39301427c0 0x7f39300cb4f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f392400d8c0 tx=0x7f3924008040 comp rx=0 tx=0).stop 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f3934c4b700 1 -- 172.21.15.146:0/277058222 reap_dead start 2024-08-15T17:03:06.455 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 shutdown_connections 2024-08-15T17:03:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 >> 172.21.15.146:0/277058222 conn(0x7f39300cdea0 msgr2=0x7f39300afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 shutdown_connections 2024-08-15T17:03:06.456 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.442+0000 7f393544c700 1 -- 172.21.15.146:0/277058222 wait complete. 2024-08-15T17:03:06.465 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-15T17:03:06.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 Processor -- start 2024-08-15T17:03:06.573 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 -- start start 2024-08-15T17:03:06.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f946c155d20 0x7f946c0b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 0x7f946c0aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.575 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0af2b0 con 0x7f946c06df10 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0adc10 con 0x7f946c155d20 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f9470a84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0af590 con 0x7f946c0ad7c0 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946bfff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f946c06df10 0x7f946c1599c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41934/0 (socket says 172.21.15.146:41934) 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946bfff700 1 -- 172.21.15.146:0/4064027781 learned_addr learned my addr 172.21.15.146:0/4064027781 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946affd700 1 --2- 172.21.15.146:0/4064027781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 0x7f946c0aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946b7fe700 1 --2- 172.21.15.146:0/4064027781 >> v2:172.21.15.161:3300/0 conn(0x7f946c155d20 0x7f946c0b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946a7fc700 1 -- 172.21.15.146:0/4064027781 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871114822 0 0) 0x7f946c0af2b0 con 0x7f946c06df10 2024-08-15T17:03:06.576 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946a7fc700 1 -- 172.21.15.146:0/4064027781 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003570 con 0x7f946c06df10 2024-08-15T17:03:06.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946affd700 1 -- 172.21.15.146:0/4064027781 >> v2:172.21.15.161:3300/0 conn(0x7f946c155d20 msgr2=0x7f946c0b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:06.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946affd700 1 --2- 172.21.15.146:0/4064027781 >> v2:172.21.15.161:3300/0 conn(0x7f946c155d20 0x7f946c0b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946affd700 1 -- 172.21.15.146:0/4064027781 >> v1:172.21.15.146:6789/0 conn(0x7f946c06df10 legacy=0x7f946c1599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.562+0000 7f946affd700 1 -- 172.21.15.146:0/4064027781 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0af870 con 0x7f946c0ad7c0 2024-08-15T17:03:06.577 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f946affd700 1 --2- 172.21.15.146:0/4064027781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 0x7f946c0aea80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e60 tx=0x7f9454009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b9b5305b69ab99 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.578 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f946a7fc700 1 -- 172.21.15.146:0/4064027781 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400e070 con 0x7f946c0ad7c0 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f946a7fc700 1 -- 172.21.15.146:0/4064027781 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400bab0 con 0x7f946c0ad7c0 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f946a7fc700 1 -- 172.21.15.146:0/4064027781 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400be30 con 0x7f946c0ad7c0 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- 172.21.15.146:0/4064027781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 msgr2=0x7f946c0aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 --2- 172.21.15.146:0/4064027781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 0x7f946c0aea80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9454009e60 tx=0x7f9454009780 comp rx=0 tx=0).stop 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- 172.21.15.146:0/4064027781 shutdown_connections 2024-08-15T17:03:06.579 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 --2- 172.21.15.146:0/4064027781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c0ad7c0 0x7f946c0aea80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 --2- 172.21.15.146:0/4064027781 >> v2:172.21.15.161:3300/0 conn(0x7f946c155d20 0x7f946c0b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- 172.21.15.146:0/4064027781 >> 172.21.15.146:0/4064027781 conn(0x7f946c0678d0 msgr2=0x7f946c067cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- 172.21.15.146:0/4064027781 shutdown_connections 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- 172.21.15.146:0/4064027781 wait complete. 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 Processor -- start 2024-08-15T17:03:06.580 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.566+0000 7f9470a84700 1 -- start start 2024-08-15T17:03:06.582 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c06df10 0x7f946c1432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c155d20 0x7f946c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c12de70 con 0x7f946c0ad7c0 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c12d260 con 0x7f946c06df10 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c12d540 con 0x7f946c155d20 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.583 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33762/0 (socket says 172.21.15.146:33762) 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 -- 172.21.15.146:0/3703687520 learned_addr learned my addr 172.21.15.146:0/3703687520 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946b7fe700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c06df10 0x7f946c1432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946bfff700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c155d20 0x7f946c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 -- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c155d20 msgr2=0x7f946c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c155d20 0x7f946c158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 -- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c06df10 msgr2=0x7f946c1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.584 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c06df10 0x7f946c1432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 -- 172.21.15.146:0/3703687520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454009a20 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f946affd700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f94540097c0 tx=0x7f9454019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454012ba0 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454012d20 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c12d820 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94540276c0 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c12cf00 con 0x7f946c0ad7c0 2024-08-15T17:03:06.585 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.570+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c12e020 con 0x7f946c0ad7c0 2024-08-15T17:03:06.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.574+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f9454024070 con 0x7f946c0ad7c0 2024-08-15T17:03:06.586 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.574+0000 7f94637fe700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 0x7f9450062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.574+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f945407ed50 con 0x7f946c0ad7c0 2024-08-15T17:03:06.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.574+0000 7f946b7fe700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 0x7f9450062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.587 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.574+0000 7f946b7fe700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 0x7f9450062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f946c0b1580 tx=0x7f94640091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.592 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.582+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f946c12e020 con 0x7f946c0ad7c0 2024-08-15T17:03:06.756 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f946c0643a0 con 0x7f9450060860 2024-08-15T17:03:06.757 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f94637fe700 1 -- 172.21.15.146:0/3703687520 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f946c0643a0 con 0x7f9450060860 2024-08-15T17:03:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 msgr2=0x7f9450062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 0x7f9450062d10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f946c0b1580 tx=0x7f94640091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 msgr2=0x7f946c12c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f94540097c0 tx=0x7f9454019000 comp rx=0 tx=0).stop 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 shutdown_connections 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f9450060860 0x7f9450062d10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f946c155d20 0x7f946c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f946c0ad7c0 0x7f946c12c9c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 --2- 172.21.15.146:0/3703687520 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f946c06df10 0x7f946c1432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.746+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 >> 172.21.15.146:0/3703687520 conn(0x7f946c0678d0 msgr2=0x7f946c148b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.760 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.750+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 shutdown_connections 2024-08-15T17:03:06.761 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.750+0000 7f9470a84700 1 -- 172.21.15.146:0/3703687520 wait complete. 2024-08-15T17:03:06.761 INFO:teuthology.orchestra.run.smithi146.stderr:instructed osd(s) 7 to deep-scrub 2024-08-15T17:03:06.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:03:06.878 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.866+0000 7f16ee650700 1 Processor -- start 2024-08-15T17:03:06.879 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.866+0000 7f16ee650700 1 -- start start 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e8159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80c9e00 con 0x7f16e8153a40 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80cc890 con 0x7f16e815a090 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80ccd20 con 0x7f16e80c9820 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e7fff700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f16e8153a40 0x7f16e80b29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41946/0 (socket says 172.21.15.146:41946) 2024-08-15T17:03:06.880 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e7fff700 1 -- 172.21.15.146:0/2261658160 learned_addr learned my addr 172.21.15.146:0/2261658160 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:06.881 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16df7fe700 1 --2- 172.21.15.146:0/2261658160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e8159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.882 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676767655 0 0) 0x7f16e80c9e00 con 0x7f16e8153a40 2024-08-15T17:03:06.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d4003570 con 0x7f16e8153a40 2024-08-15T17:03:06.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16df7fe700 1 -- 172.21.15.146:0/2261658160 >> v1:172.21.15.161:6789/0 conn(0x7f16e815a090 legacy=0x7f16e80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 <== mon.0 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042113051 0 0) 0x7f16d4003570 con 0x7f16e8153a40 2024-08-15T17:03:06.883 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16df7fe700 1 -- 172.21.15.146:0/2261658160 >> v1:172.21.15.146:6789/0 conn(0x7f16e8153a40 legacy=0x7f16e80b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16df7fe700 1 -- 172.21.15.146:0/2261658160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e815a4b0 con 0x7f16e80c9820 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16df7fe700 1 --2- 172.21.15.146:0/2261658160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e8159b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009910 tx=0x7f16d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633c27a077ac43ee server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d8001470 con 0x7f16e80c9820 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0011420 con 0x7f16e80c9820 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16e77fe700 1 -- 172.21.15.146:0/2261658160 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00117a0 con 0x7f16e80c9820 2024-08-15T17:03:06.884 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- 172.21.15.146:0/2261658160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 msgr2=0x7f16e8159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 --2- 172.21.15.146:0/2261658160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e8159b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009910 tx=0x7f16d00091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- 172.21.15.146:0/2261658160 shutdown_connections 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 --2- 172.21.15.146:0/2261658160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e8159b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- 172.21.15.146:0/2261658160 >> 172.21.15.146:0/2261658160 conn(0x7f16e80a9db0 msgr2=0x7f16e80ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- 172.21.15.146:0/2261658160 shutdown_connections 2024-08-15T17:03:06.885 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.870+0000 7f16ee650700 1 -- 172.21.15.146:0/2261658160 wait complete. 2024-08-15T17:03:06.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 Processor -- start 2024-08-15T17:03:06.887 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- start start 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e814a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 0x7f16e8134350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e809a280 con 0x7f16e815a090 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80985d0 con 0x7f16e8153a40 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80979c0 con 0x7f16e80c9820 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33768/0 (socket says 172.21.15.146:33768) 2024-08-15T17:03:06.888 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 -- 172.21.15.146:0/3038710939 learned_addr learned my addr 172.21.15.146:0/3038710939 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16e7fff700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e814a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16df7fe700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 0x7f16e8134350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 -- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 msgr2=0x7f16e814a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e814a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 -- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 msgr2=0x7f16e8134350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 0x7f16e8134350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 -- 172.21.15.146:0/3038710939 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d0009420 con 0x7f16e815a090 2024-08-15T17:03:06.889 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16df7fe700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 0x7f16e8134350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ecbed700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16e000cb90 tx=0x7f16e00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e000a7e0 con 0x7f16e815a090 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e000a960 con 0x7f16e815a090 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e8097ca0 con 0x7f16e815a090 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e000ad20 con 0x7f16e815a090 2024-08-15T17:03:06.890 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.874+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e8098260 con 0x7f16e815a090 2024-08-15T17:03:06.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.878+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e8098760 con 0x7f16e815a090 2024-08-15T17:03:06.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.878+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f16e0003030 con 0x7f16e815a090 2024-08-15T17:03:06.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.878+0000 7f16e4ff9700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 0x7f16c8062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:06.891 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.878+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f16e007ca50 con 0x7f16e815a090 2024-08-15T17:03:06.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.878+0000 7f16e7fff700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 0x7f16c8062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:06.892 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.882+0000 7f16e7fff700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 0x7f16c8062d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8001a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:06.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:06.886+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f16e004d140 con 0x7f16e815a090 2024-08-15T17:03:07.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.046+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16e80645c0 con 0x7f16c8060890 2024-08-15T17:03:07.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16e4ff9700 1 -- 172.21.15.146:0/3038710939 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48496 (secure 0 0 0) 0x7f16e80645c0 con 0x7f16c8060890 2024-08-15T17:03:07.065 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:07.066 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 msgr2=0x7f16c8062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 0x7f16c8062d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16d8000c00 tx=0x7f16d8001a20 comp rx=0 tx=0).stop 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 msgr2=0x7f16e806b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16e000cb90 tx=0x7f16e00017a0 comp rx=0 tx=0).stop 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 shutdown_connections 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f16c8060890 0x7f16c8062d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:07.067 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16e815a090 0x7f16e806b8f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:07.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16e8153a40 0x7f16e8134350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:07.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 --2- 172.21.15.146:0/3038710939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16e80c9820 0x7f16e814a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:07.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 >> 172.21.15.146:0/3038710939 conn(0x7f16e80a9db0 msgr2=0x7f16e8154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:07.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 shutdown_connections 2024-08-15T17:03:07.068 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:07.054+0000 7f16ee650700 1 -- 172.21.15.146:0/3038710939 wait complete. 2024-08-15T17:03:07.069 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:03:07.080 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-08-15T17:03:05.722926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2396464,"kb_used_data":5124,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578256,"statfs":{"total":773094113280,"available":770640134144,"internally_reserved":0,"allocated":5246976,"data_stored":2867335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.135772"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":181,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-08-15T17:02:57.590203+0000","last_change":"2024-08-15T16:59:37.683222+0000","last_active":"2024-08-15T17:02:57.590203+0000","last_peered":"2024-08-15T17:02:57.590203+0000","last_clean":"2024-08-15T17:02:57.590203+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:02:57.590203+0000","last_undegraded":"2024-08-15T17:02:57.590203+0000","last_fullsized":"2024-08-15T17:02:57.590203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-15T16:59:36.359214+0000","last_clean_scrub_stamp":"2024-08-15T16:59:36.359214+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T20:47:50.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298432,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073408,"statfs":{"total":96636764160,"available":96331169792,"internally_reserved":0,"allocated":1101824,"data_stored":801313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86699999999999999,"15min":0.83199999999999996},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.2430000000000001,"5min":1.258,"15min":1.258},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.335,"15min":1.335},"last":0.99199999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.169,"5min":1.415,"15min":1.415},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1479999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.075}]},{"osd":2,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.69099999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2230000000000001,"5min":1.329,"15min":1.329},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.105},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.518,"15min":1.518},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97699999999999998,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.264,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1,"15min":0.98999999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2430000000000001,"5min":1.292,"15min":1.292},"last":0.79900000000000004}]},{"osd":4,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.63800000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.327,"5min":1.357,"15min":1.357},"last":0.873},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.048,"15min":1.056},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.304,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.024}]},{"osd":5,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1.0700000000000001},"min":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.26,"5min":1.409,"15min":1.409},"last":1.052},{"interface":"front","average":{"1min":1.012,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76200000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0649999999999999,"15min":1.079},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2210000000000001,"5min":1.387,"15min":1.387},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.93100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.381,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.93300000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.90400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95499999999999996,"15min":0.96099999999999997},"min":{"1min":0.77200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":2.097,"15min":2.097},"last":0.97799999999999998}]},{"osd":1,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.72299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.121,"5min":1.25,"15min":1.25},"last":1.069},{"interface":"front","average":{"1min":0.748,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.61099999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1180000000000001,"5min":1.444,"15min":1.444},"last":0.624}]},{"osd":2,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.379,"15min":1.379},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.121,"5min":1.373,"15min":1.373},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3580000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":1.0920000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.65900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3420000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.397,"5min":1.698,"15min":1.698},"last":1.0409999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0900000000000001,"15min":1.117},"min":{"1min":0.745,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.46,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.216,"15min":1.244},"min":{"1min":0.90000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.603,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.173}]}]},{"osd":5,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.876,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.0629999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":0.877},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1140000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.995}]},{"osd":2,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.286,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.143},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.103,"15min":1.129},"min":{"1min":0.879,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.389,"5min":1.72,"15min":1.72},"last":1.1879999999999999}]},{"osd":3,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.02,"15min":1.042},"min":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2290000000000001,"5min":1.417,"15min":1.417},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0940000000000001,"15min":1.1240000000000001},"min":{"1min":0.78400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2969999999999999,"5min":1.593,"15min":1.593},"last":1.1140000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.028,"15min":1.075},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1659999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.117,"5min":1.161,"15min":1.1850000000000001},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3560000000000001,"5min":1.889,"15min":1.889},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.133,"15min":1.1739999999999999},"min":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3420000000000001,"5min":1.734,"15min":1.734},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.075,"5min":1.1839999999999999,"15min":1.23},"min":{"1min":0.86599999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.268,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.3},"min":{"1min":1.004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4390000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":0.745},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3180000000000001,"15min":1.3600000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.4590000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0189999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.875},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2450000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":0.999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.626,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.97399999999999998}]},{"osd":1,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.048,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.749,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.2130000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.94299999999999995},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.23,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93000000000000005,"15min":0.92200000000000004},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.93700000000000006,"15min":0.94299999999999995},"min":{"1min":0.71099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.181,"5min":1.794,"15min":1.794},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.048,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.145}]},{"osd":5,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0249999999999999,"15min":1.075},"min":{"1min":0.76100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.105},{"interface":"front","average":{"1min":0.996,"5min":1.0609999999999999,"15min":1.095},"min":{"1min":0.79900000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.046}]},{"osd":6,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.075,"15min":1.101},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.333,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.244},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.504,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.278}]},{"osd":7,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.111,"15min":1.135},"min":{"1min":0.90700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2470000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":1.038,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.587,"5min":1.99,"15min":1.99},"last":1.3109999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.843,"15min":1.843},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.55700000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92400000000000004,"5min":1.202,"15min":1.202},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0129999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.65600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0289999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.748}]},{"osd":2,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0129999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.67400000000000004}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.997},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1859999999999999,"5min":1.956,"15min":1.956},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.071,"15min":1.083},"min":{"1min":0.92500000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.151,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1879999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.81799999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.218,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.077,"15min":1.101},"min":{"1min":0.69399999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.196,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0920000000000001,"15min":1.107},"min":{"1min":0.81299999999999994,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2390000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.036,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.86499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.1679999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.0369999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.90900000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.222,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0920000000000001,"15min":1.0800000000000001},"min":{"1min":0.93000000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.337,"5min":1.573,"15min":1.573},"last":1.3600000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.008,"15min":1.008},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.54000000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.66500000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.0660000000000001}]},{"osd":3,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.875},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.70799999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.036}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.82099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.26,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.022,"15min":1.012},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.448,"5min":1.734,"15min":1.734},"last":1.1739999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0189999999999999,"15min":0.997},"min":{"1min":0.72699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.008,"15min":1.018},"min":{"1min":0.79200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.173,"5min":1.234,"15min":1.234},"last":1.2050000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.339,"5min":1.375,"15min":1.375},"last":1.103},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3089999999999999,"5min":1.397,"15min":1.397},"last":1.268}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1180000000000001,"15min":1.1040000000000001},"min":{"1min":0.91200000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.246},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.98999999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.268,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.3069999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297856,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.123},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.22,"15min":1.22},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0960000000000001,"5min":1.167,"15min":1.167},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.69599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.004,"15min":1.018},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4890000000000001,"5min":1.647,"15min":1.647},"last":1.018},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.76600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4510000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1000000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.248,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.071},"min":{"1min":0.94599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.1120000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.075,"15min":1.095},"min":{"1min":0.81100000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.331,"5min":1.96,"15min":1.96},"last":1.044},{"interface":"front","average":{"1min":1.131,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.3580000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.087}]},{"osd":7,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.2529999999999999},"min":{"1min":0.90600000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.4510000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.274,"5min":1.153,"15min":1.125},"min":{"1min":1.0469999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.97199999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1097728,"data_stored":800352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.119,"15min":1.119},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0169999999999999,"5min":1.179,"15min":1.179},"last":0.44800000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.503},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.78700000000000003},"min":{"1min":0.752,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0660000000000001,"5min":1.385,"15min":1.385},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.044,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.95899999999999996,"15min":0.94899999999999995},"min":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.252,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.304,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.91600000000000004}]},{"osd":5,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.377,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.071,"15min":1.0569999999999999},"min":{"1min":0.92200000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.327,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.0720000000000001}]},{"osd":6,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.044,"15min":1.038},"min":{"1min":0.84899999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.169,"15min":1.1679999999999999},"min":{"1min":1.0489999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.385,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.125},"min":{"1min":0.96299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.335,"5min":1.877,"15min":1.877},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.165,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":1.052,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4239999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.95999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:03:07.083 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-15T16:59:36.359214+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=16, tm_min=59, tm_sec=36, tm_wday=3, tm_yday=228, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=15, tm_hour=17, tm_min=3, tm_sec=1, tm_wday=3, tm_yday=228, tm_isdst=0) 2024-08-15T17:03:07.084 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-15T17:03:17.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:17.380 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:27.084 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-15T17:03:27.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 Processor -- start 2024-08-15T17:03:27.200 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 -- start start 2024-08-15T17:03:27.201 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 0x7f01580cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580cf190 con 0x7f015815d5b0 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580ce5e0 con 0x7f01580d0010 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015da47700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580ce860 con 0x7f0158159040 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015d246700 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f015815d5b0 0x7f015815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41128/0 (socket says 172.21.15.146:41128) 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015d246700 1 -- 172.21.15.146:0/3010176195 learned_addr learned my addr 172.21.15.146:0/3010176195 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:27.202 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015ca45700 1 --2- 172.21.15.146:0/3010176195 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 0x7f01580cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 <== mon.0 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695736641 0 0) 0x7f01580cf190 con 0x7f015815d5b0 2024-08-15T17:03:27.204 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0148003570 con 0x7f015815d5b0 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571204256 0 0) 0x7f01580ce5e0 con 0x7f01580d0010 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01580cf190 con 0x7f01580d0010 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015ca45700 1 -- 172.21.15.146:0/3010176195 >> v1:172.21.15.161:6789/0 conn(0x7f01580d0010 legacy=0x7f015815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015ca45700 1 -- 172.21.15.146:0/3010176195 >> v1:172.21.15.146:6789/0 conn(0x7f015815d5b0 legacy=0x7f015815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015ca45700 1 -- 172.21.15.146:0/3010176195 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580ceb40 con 0x7f0158159040 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f015ca45700 1 --2- 172.21.15.146:0/3010176195 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 0x7f01580cf620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f014c009910 tx=0x7f014c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3d66734b2ccb9fb server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c011630 con 0x7f0158159040 2024-08-15T17:03:27.205 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c011790 con 0x7f0158159040 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.189+0000 7f01577fe700 1 -- 172.21.15.146:0/3010176195 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c011b10 con 0x7f0158159040 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- 172.21.15.146:0/3010176195 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 msgr2=0x7f01580cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 --2- 172.21.15.146:0/3010176195 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 0x7f01580cf620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f014c009910 tx=0x7f014c0091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- 172.21.15.146:0/3010176195 shutdown_connections 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 --2- 172.21.15.146:0/3010176195 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0158159040 0x7f01580cf620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- 172.21.15.146:0/3010176195 >> 172.21.15.146:0/3010176195 conn(0x7f0158068250 msgr2=0x7f0158068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- 172.21.15.146:0/3010176195 shutdown_connections 2024-08-15T17:03:27.206 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- 172.21.15.146:0/3010176195 wait complete. 2024-08-15T17:03:27.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 Processor -- start 2024-08-15T17:03:27.207 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.193+0000 7f015da47700 1 -- start start 2024-08-15T17:03:27.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01580d0010 0x7f01580b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0158159040 0x7f0158153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015814e790 con 0x7f015815d5b0 2024-08-15T17:03:27.208 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01580b23a0 con 0x7f0158159040 2024-08-15T17:03:27.209 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580b2880 con 0x7f01580d0010 2024-08-15T17:03:27.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33684/0 (socket says 172.21.15.146:33684) 2024-08-15T17:03:27.210 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015ca45700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01580d0010 0x7f01580b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 -- 172.21.15.146:0/2872091388 learned_addr learned my addr 172.21.15.146:0/2872091388 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f0157fff700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0158159040 0x7f0158153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 -- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01580d0010 msgr2=0x7f01580b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01580d0010 0x7f01580b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 -- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0158159040 msgr2=0x7f0158153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0158159040 0x7f0158153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.211 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 -- 172.21.15.146:0/2872091388 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c009420 con 0x7f015815d5b0 2024-08-15T17:03:27.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015d246700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f015000cd00 tx=0x7f0150001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015000ce00 con 0x7f015815d5b0 2024-08-15T17:03:27.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f015000aac0 con 0x7f015815d5b0 2024-08-15T17:03:27.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01580b1a40 con 0x7f015815d5b0 2024-08-15T17:03:27.212 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015000ae80 con 0x7f015815d5b0 2024-08-15T17:03:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.197+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01580b2000 con 0x7f015815d5b0 2024-08-15T17:03:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01580b2520 con 0x7f015815d5b0 2024-08-15T17:03:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f0150007300 con 0x7f015815d5b0 2024-08-15T17:03:27.213 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f0154ff9700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 0x7f013c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f015007cc20 con 0x7f015815d5b0 2024-08-15T17:03:27.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f015ca45700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 0x7f013c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.201+0000 7f015ca45700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 0x7f013c062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f014c0091e0 tx=0x7f014c017000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.218 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.205+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f01580b2520 con 0x7f015815d5b0 2024-08-15T17:03:27.372 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.361+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 --> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01580b24e0 con 0x7f013c060860 2024-08-15T17:03:27.378 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.365+0000 7f0154ff9700 1 -- 172.21.15.146:0/2872091388 <== mgr.4102 v2:172.21.15.161:6832/24128 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48340 (secure 0 0 0) 0x7f01580b24e0 con 0x7f013c060860 2024-08-15T17:03:27.379 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:27.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 msgr2=0x7f013c062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 0x7f013c062d10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f014c0091e0 tx=0x7f014c017000 comp rx=0 tx=0).stop 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 msgr2=0x7f0158149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f015000cd00 tx=0x7f0150001ac0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 shutdown_connections 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f013c060860 0x7f013c062d10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.381 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f015815d5b0 0x7f0158149860 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0158159040 0x7f0158153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 --2- 172.21.15.146:0/2872091388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01580d0010 0x7f01580b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 >> 172.21.15.146:0/2872091388 conn(0x7f0158068250 msgr2=0x7f015815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:27.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 shutdown_connections 2024-08-15T17:03:27.382 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.369+0000 7f015da47700 1 -- 172.21.15.146:0/2872091388 wait complete. 2024-08-15T17:03:27.383 INFO:teuthology.orchestra.run.smithi146.stderr:dumped all 2024-08-15T17:03:27.394 INFO:teuthology.orchestra.run.smithi146.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-08-15T17:03:25.728288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2396456,"kb_used_data":5124,"kb_used_omap":13,"kb_used_meta":2391218,"kb_avail":752578264,"statfs":{"total":773094113280,"available":770640142336,"internally_reserved":0,"allocated":5246976,"data_stored":2867335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2448607936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003146"},"pg_stats":[{"pgid":"1.0","version":"43'119","reported_seq":194,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-08-15T17:03:12.886268+0000","last_change":"2024-08-15T17:03:12.886229+0000","last_active":"2024-08-15T17:03:12.886268+0000","last_peered":"2024-08-15T17:03:12.886268+0000","last_clean":"2024-08-15T17:03:12.886268+0000","last_became_active":"2024-08-15T16:59:37.680940+0000","last_became_peered":"2024-08-15T16:59:37.680940+0000","last_unstale":"2024-08-15T17:03:12.886268+0000","last_undegraded":"2024-08-15T17:03:12.886268+0000","last_fullsized":"2024-08-15T17:03:12.886268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'119","last_scrub_stamp":"2024-08-15T17:03:12.886087+0000","last_deep_scrub":"43'119","last_deep_scrub_stamp":"2024-08-15T17:03:12.886087+0000","last_clean_scrub_stamp":"2024-08-15T17:03:12.886087+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-16T17:37:30.905120+0000","scrub_duration":0.011631786,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542193,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298428,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073412,"statfs":{"total":96636764160,"available":96331173888,"internally_reserved":0,"allocated":1101824,"data_stored":801313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.86699999999999999,"15min":0.83199999999999996},"min":{"1min":0.73499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.2430000000000001,"5min":1.258,"15min":1.258},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.86699999999999999},"min":{"1min":0.64900000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.335,"15min":1.335},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.94899999999999995,"15min":0.93899999999999995},"min":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.169,"5min":1.415,"15min":1.415},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93799999999999994,"15min":0.92400000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1479999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.1180000000000001}]},{"osd":2,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.69099999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.2230000000000001,"5min":1.329,"15min":1.329},"last":1.073},{"interface":"front","average":{"1min":1.089,"5min":1.101,"15min":1.105},"min":{"1min":0.79900000000000004,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2989999999999999,"5min":1.518,"15min":1.518},"last":0.88}]},{"osd":3,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.97699999999999998,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.264,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1,"15min":0.98999999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2430000000000001,"5min":1.292,"15min":1.292},"last":1.04}]},{"osd":4,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.89000000000000001,"15min":0.88700000000000001},"min":{"1min":0.63800000000000001,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.327,"5min":1.357,"15min":1.357},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.048,"15min":1.056},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.304,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.153}]},{"osd":5,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":1.044,"15min":1.0700000000000001},"min":{"1min":0.70799999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.26,"5min":1.409,"15min":1.409},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.3009999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.1799999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0649999999999999,"15min":1.079},"min":{"1min":0.76000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.2210000000000001,"5min":1.387,"15min":1.387},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.1699999999999999,"15min":1.153},"min":{"1min":0.93100000000000005,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.381,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.91200000000000003,"15min":0.90400000000000003},"min":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.196,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.95499999999999996,"15min":0.96099999999999997},"min":{"1min":0.77200000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2050000000000001,"5min":2.097,"15min":2.097},"last":0.82499999999999996}]},{"osd":1,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.72299999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.121,"5min":1.25,"15min":1.25},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.872,"15min":0.89200000000000002},"min":{"1min":0.61099999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1180000000000001,"5min":1.444,"15min":1.444},"last":0.73699999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93999999999999995,"15min":0.94499999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.264,"5min":1.379,"15min":1.379},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.92700000000000005,"15min":0.93899999999999995},"min":{"1min":0.63100000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.121,"5min":1.373,"15min":1.373},"last":0.877}]},{"osd":3,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.86799999999999999},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.98399999999999999,"15min":0.98799999999999999},"min":{"1min":0.66700000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3580000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.95899999999999996,"15min":0.97099999999999997},"min":{"1min":0.59899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.02},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.123,"15min":1.1359999999999999},"min":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.423,"5min":1.645,"15min":1.645},"last":1.3600000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.1140000000000001,"15min":1.1439999999999999},"min":{"1min":0.65900000000000003,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.3420000000000001,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1639999999999999,"15min":1.1779999999999999},"min":{"1min":0.999,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.397,"5min":1.698,"15min":1.698},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:44 2024","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0900000000000001,"15min":1.117},"min":{"1min":0.745,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.46,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.3},{"interface":"front","average":{"1min":1.151,"5min":1.216,"15min":1.244},"min":{"1min":0.90000000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.603,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.105}]}]},{"osd":5,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.876,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0149999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.93700000000000006,"15min":0.94099999999999995},"min":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2}]},{"osd":1,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.2470000000000001,"5min":1.298,"15min":1.298},"last":1.177},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.01,"15min":1.04},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1140000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0669999999999999,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.286,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.103,"15min":1.129},"min":{"1min":0.879,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.389,"5min":1.72,"15min":1.72},"last":0.82399999999999995}]},{"osd":3,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.02,"15min":1.042},"min":{"1min":0.77100000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2290000000000001,"5min":1.417,"15min":1.417},"last":1.04},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0940000000000001,"15min":1.1240000000000001},"min":{"1min":0.78400000000000003,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.2969999999999999,"5min":1.593,"15min":1.593},"last":0.96499999999999997}]},{"osd":4,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.028,"15min":1.075},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.1659999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.995},{"interface":"front","average":{"1min":1.117,"5min":1.161,"15min":1.1850000000000001},"min":{"1min":0.751,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3560000000000001,"5min":1.889,"15min":1.889},"last":1.135}]},{"osd":6,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.133,"15min":1.1739999999999999},"min":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.3420000000000001,"5min":1.734,"15min":1.734},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.075,"5min":1.1839999999999999,"15min":1.23},"min":{"1min":0.86599999999999999,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.268,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.075}]},{"osd":7,"last update":"Thu Aug 15 17:02:46 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2629999999999999,"15min":1.3},"min":{"1min":1.004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4390000000000001,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3180000000000001,"15min":1.3600000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"max":{"1min":1.4590000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2250000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542192,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302016,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069824,"statfs":{"total":96636764160,"available":96327499776,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.89900000000000002,"15min":0.875},"min":{"1min":0.69699999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2450000000000001,"5min":1.9370000000000001,"15min":1.9370000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.626,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.872,"15min":0.89000000000000001},"min":{"1min":0.64300000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.101},{"interface":"front","average":{"1min":1.048,"5min":0.93600000000000005,"15min":0.90600000000000003},"min":{"1min":0.749,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0009999999999999}]},{"osd":2,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95299999999999996,"15min":0.94299999999999995},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.23,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.179},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.93000000000000005,"15min":0.92200000000000004},"min":{"1min":0.749,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.93700000000000006,"15min":0.94299999999999995},"min":{"1min":0.71099999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.181,"5min":1.794,"15min":1.794},"last":0.878},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.048,"15min":1.069},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.198,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0249999999999999,"15min":1.075},"min":{"1min":0.76100000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.206},{"interface":"front","average":{"1min":0.996,"5min":1.0609999999999999,"15min":1.095},"min":{"1min":0.79900000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.3049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.0389999999999999}]},{"osd":6,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.075,"15min":1.101},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.333,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.167,"15min":1.181},"min":{"1min":0.94799999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.504,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.149}]},{"osd":7,"last update":"Thu Aug 15 17:02:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.111,"15min":1.135},"min":{"1min":0.90700000000000003,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.2470000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.127},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.236,"15min":1.2569999999999999},"min":{"1min":1.038,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":1.587,"5min":1.99,"15min":1.99},"last":1.2350000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65400000000000003,"15min":0.67300000000000004},"min":{"1min":0.52300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1200000000000001,"5min":1.843,"15min":1.843},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.55700000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92400000000000004,"5min":1.202,"15min":1.202},"last":0.51300000000000001}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76500000000000001,"15min":0.76300000000000001},"min":{"1min":0.52400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.0129999999999999,"5min":2.0649999999999999,"15min":2.0649999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83299999999999996,"15min":0.82999999999999996},"min":{"1min":0.65600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0289999999999999,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":1.141}]},{"osd":2,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77300000000000002,"15min":0.78900000000000003},"min":{"1min":0.61099999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.89300000000000002,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84999999999999998,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0129999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.80400000000000005}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.997},"min":{"1min":0.624,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.1859999999999999,"5min":1.956,"15min":1.956},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.071,"15min":1.083},"min":{"1min":0.92500000000000004,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.151,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.95299999999999996}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.81799999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.218,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.044},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.077,"15min":1.101},"min":{"1min":0.69399999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.196,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.02}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.0920000000000001,"15min":1.107},"min":{"1min":0.81299999999999994,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.2390000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.86499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.1679999999999999,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.089}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.121,"15min":1.1279999999999999},"min":{"1min":0.90900000000000003,"5min":0.752,"15min":0.752},"max":{"1min":1.222,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0920000000000001,"15min":1.0800000000000001},"min":{"1min":0.93000000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.337,"5min":1.573,"15min":1.573},"last":1.119}]}]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297852,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073988,"statfs":{"total":96636764160,"available":96331763712,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.65800000000000003},"min":{"1min":0.503,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91600000000000004,"5min":1.008,"15min":1.008},"last":0.502},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.67000000000000004},"min":{"1min":0.54000000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.878,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.66500000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85499999999999998,"15min":0.84299999999999997},"min":{"1min":0.70799999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.746}]},{"osd":4,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.998,"15min":0.99399999999999999},"min":{"1min":0.82099999999999995,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.26,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.022,"15min":1.012},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.448,"5min":1.734,"15min":1.734},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0189999999999999,"15min":0.997},"min":{"1min":0.72699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.008,"15min":1.018},"min":{"1min":0.79200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.173,"5min":1.234,"15min":1.234},"last":1.113}]},{"osd":6,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.77300000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.339,"5min":1.375,"15min":1.375},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.1000000000000001,"15min":1.1040000000000001},"min":{"1min":0.84599999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.3089999999999999,"5min":1.397,"15min":1.397},"last":0.94099999999999995}]},{"osd":7,"last update":"Thu Aug 15 17:02:37 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1180000000000001,"15min":1.1040000000000001},"min":{"1min":0.91200000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":1.012},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.98999999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.268,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542193,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297856,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073984,"statfs":{"total":96636764160,"available":96331759616,"internally_reserved":0,"allocated":507904,"data_stored":210945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68799999999999994,"15min":0.69499999999999995},"min":{"1min":0.51900000000000002,"5min":0.505,"15min":0.505},"max":{"1min":0.94599999999999995,"5min":1.123,"15min":1.123},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.70799999999999996},"min":{"1min":0.53000000000000003,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0660000000000001,"5min":1.22,"15min":1.22},"last":0.83199999999999996}]},{"osd":2,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68300000000000005,"15min":0.66900000000000004},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.61199999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97399999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.79700000000000004},"min":{"1min":0.64200000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0960000000000001,"5min":1.167,"15min":1.167},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.69599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.88800000000000001}]},{"osd":4,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":1.004,"15min":1.018},"min":{"1min":0.79700000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4890000000000001,"5min":1.647,"15min":1.647},"last":1.129},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0129999999999999,"15min":1.0089999999999999},"min":{"1min":0.76600000000000001,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.4510000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.1000000000000001,"15min":1.121},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.248,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.056},{"interface":"front","average":{"1min":1.133,"5min":1.0860000000000001,"15min":1.071},"min":{"1min":0.94599999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.036}]},{"osd":6,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.075,"15min":1.095},"min":{"1min":0.81100000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.331,"5min":1.96,"15min":1.96},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1870000000000001,"15min":1.21},"min":{"1min":0.94999999999999996,"5min":0.745,"15min":0.745},"max":{"1min":1.3580000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":1.341}]},{"osd":7,"last update":"Thu Aug 15 17:02:36 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2290000000000001,"15min":1.2529999999999999},"min":{"1min":0.90600000000000003,"5min":0.87,"15min":0.87},"max":{"1min":1.4510000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.228},{"interface":"front","average":{"1min":1.274,"5min":1.153,"15min":1.125},"min":{"1min":1.0469999999999999,"5min":0.873,"15min":0.873},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0840000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302584,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1097728,"data_stored":800352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66700000000000004,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.056,"5min":1.119,"15min":1.119},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.51500000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.0169999999999999,"5min":1.179,"15min":1.179},"last":0.66100000000000003}]},{"osd":2,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.58799999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0640000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.81399999999999995,"15min":0.78700000000000003},"min":{"1min":0.752,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0660000000000001,"5min":1.385,"15min":1.385},"last":0.995}]},{"osd":3,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.754,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.99399999999999999,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.71499999999999997,"5min":0.497,"15min":0.497},"max":{"1min":1.044,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.04}]},{"osd":4,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.95899999999999996,"15min":0.94899999999999995},"min":{"1min":0.69699999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.252,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.014},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.304,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.0840000000000001}]},{"osd":5,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0960000000000001,"15min":1.0920000000000001},"min":{"1min":0.871,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.377,"5min":1.5860000000000001,"15min":1.5860000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.071,"15min":1.0569999999999999},"min":{"1min":0.92200000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.327,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.129}]},{"osd":6,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.044,"15min":1.038},"min":{"1min":0.84899999999999998,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.323},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.169,"15min":1.1679999999999999},"min":{"1min":1.0489999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"max":{"1min":1.385,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1659999999999999}]},{"osd":7,"last update":"Thu Aug 15 17:02:38 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1319999999999999,"15min":1.125},"min":{"1min":0.96299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.335,"5min":1.877,"15min":1.877},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.1259999999999999,"15min":1.123},"min":{"1min":1.052,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.4239999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2410000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-15T17:03:27.396 DEBUG:teuthology.orchestra.run.smithi146:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-15T17:03:27.503 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 Processor -- start 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 -- start start 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 0x7f43b0157940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43b00b0d90 0x7f43b00b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b006e010 con 0x7f43b01514d0 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00b1750 con 0x7f43b00b0d90 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43af59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0157fb0 con 0x7f43b01574f0 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43add9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 0x7f43b0157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.504 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43aed9d700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f43b00b0d90 0x7f43b00b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43ae59c700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f43b01514d0 0x7f43b006ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:33386/0 (socket says 172.21.15.146:33386) 2024-08-15T17:03:27.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43add9b700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 0x7f43b0157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:40896/0 (socket says 172.21.15.146:40896) 2024-08-15T17:03:27.505 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.489+0000 7f43ae59c700 1 -- 172.21.15.146:0/3640886759 learned_addr learned my addr 172.21.15.146:0/3640886759 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:27.507 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43aed9d700 1 -- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 msgr2=0x7f43b0157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43aed9d700 1 --2- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 0x7f43b0157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43aed9d700 1 -- 172.21.15.146:0/3640886759 >> v1:172.21.15.161:6789/0 conn(0x7f43b01514d0 legacy=0x7f43b006ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43aed9d700 1 -- 172.21.15.146:0/3640886759 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0158290 con 0x7f43b00b0d90 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43aed9d700 1 --2- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b00b1210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009e60 tx=0x7f43a8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba8ba357612f7d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43ad59a700 1 -- 172.21.15.146:0/3640886759 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e070 con 0x7f43b00b0d90 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43ad59a700 1 -- 172.21.15.146:0/3640886759 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800ba60 con 0x7f43b00b0d90 2024-08-15T17:03:27.508 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43ad59a700 1 -- 172.21.15.146:0/3640886759 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800be00 con 0x7f43b00b0d90 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 -- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 msgr2=0x7f43b00b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 --2- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b00b1210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009e60 tx=0x7f43a8009780 comp rx=0 tx=0).stop 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 -- 172.21.15.146:0/3640886759 shutdown_connections 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 --2- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b00b1210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 --2- 172.21.15.146:0/3640886759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01574f0 0x7f43b0157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 -- 172.21.15.146:0/3640886759 >> 172.21.15.146:0/3640886759 conn(0x7f43b00678a0 msgr2=0x7f43b0067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:27.509 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 -- 172.21.15.146:0/3640886759 shutdown_connections 2024-08-15T17:03:27.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.493+0000 7f43af59e700 1 -- 172.21.15.146:0/3640886759 wait complete. 2024-08-15T17:03:27.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 Processor -- start 2024-08-15T17:03:27.510 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- start start 2024-08-15T17:03:27.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01514d0 0x7f43b0131340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.512 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0069410 con 0x7f43b01574f0 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00696f0 con 0x7f43b00b0d90 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00699d0 con 0x7f43b01514d0 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:33694/0 (socket says 172.21.15.146:33694) 2024-08-15T17:03:27.513 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 -- 172.21.15.146:0/3736768581 learned_addr learned my addr 172.21.15.146:0/3736768581 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43add9b700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01514d0 0x7f43b0131340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43ae59c700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 -- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01514d0 msgr2=0x7f43b0131340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01514d0 0x7f43b0131340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 -- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 msgr2=0x7f43b006a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.514 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b006a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 -- 172.21.15.146:0/3736768581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009a20 con 0x7f43b01574f0 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43ae59c700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b006a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43aed9d700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b6c0 tx=0x7f43a8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8012e20 con 0x7f43b01574f0 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8026460 con 0x7f43b01574f0 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b0069cb0 con 0x7f43b01574f0 2024-08-15T17:03:27.515 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8026820 con 0x7f43b01574f0 2024-08-15T17:03:27.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.497+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b006a9f0 con 0x7f43b01574f0 2024-08-15T17:03:27.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b0064330 con 0x7f43b01574f0 2024-08-15T17:03:27.516 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f43a801d420 con 0x7f43b01574f0 2024-08-15T17:03:27.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f439e7fc700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 0x7f4390062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f43a807ee00 con 0x7f43b01574f0 2024-08-15T17:03:27.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f43ae59c700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 0x7f4390062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.517 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.501+0000 7f43ae59c700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 0x7f4390062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43a4007fd0 tx=0x7f43a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.523 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.509+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f43a804f450 con 0x7f43b01574f0 2024-08-15T17:03:27.687 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.677+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f43b0063980 con 0x7f43b01574f0 2024-08-15T17:03:27.696 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.685+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f43a8009850 con 0x7f43b01574f0 2024-08-15T17:03:27.697 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.685+0000 7f439e7fc700 1 -- 172.21.15.146:0/3736768581 <== mon.0 v2:172.21.15.146:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f43a80210c0 con 0x7f43b01574f0 2024-08-15T17:03:27.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 msgr2=0x7f4390062d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 0x7f4390062d10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43a4007fd0 tx=0x7f43a4009040 comp rx=0 tx=0).stop 2024-08-15T17:03:27.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 msgr2=0x7f43b0131880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b6c0 tx=0x7f43a8018000 comp rx=0 tx=0).stop 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 shutdown_connections 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f4390060860 0x7f4390062d10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43b01574f0 0x7f43b0131880 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f43b01514d0 0x7f43b0131340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 --2- 172.21.15.146:0/3736768581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43b00b0d90 0x7f43b006a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 >> 172.21.15.146:0/3736768581 conn(0x7f43b00678a0 msgr2=0x7f43b00ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:27.702 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 shutdown_connections 2024-08-15T17:03:27.703 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.689+0000 7f43af59e700 1 -- 172.21.15.146:0/3736768581 wait complete. 2024-08-15T17:03:27.713 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-15T17:03:27.714 DEBUG:teuthology.orchestra.run.smithi146:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-15T17:03:27.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 Processor -- start 2024-08-15T17:03:27.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 -- start start 2024-08-15T17:03:27.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 0x7f55140ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.821 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 0x7f5514157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55140adf50 con 0x7f5514157890 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140ae920 con 0x7f5514156f80 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5519e19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55140ad340 con 0x7f55140ace80 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 0x7f5514157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5513fff700 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f5514157890 0x7f55140aab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.146:33402/0 (socket says 172.21.15.146:33402) 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f55137fe700 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 0x7f55140ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5513fff700 1 -- 172.21.15.146:0/2474941583 learned_addr learned my addr 172.21.15.146:0/2474941583 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-08-15T17:03:27.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5512ffd700 1 -- 172.21.15.146:0/2474941583 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782374616 0 0) 0x7f55140adf50 con 0x7f5514157890 2024-08-15T17:03:27.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5512ffd700 1 -- 172.21.15.146:0/2474941583 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5500003570 con 0x7f5514157890 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 -- 172.21.15.146:0/2474941583 >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 msgr2=0x7f55140ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 --2- 172.21.15.146:0/2474941583 >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 0x7f55140ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 -- 172.21.15.146:0/2474941583 >> v1:172.21.15.161:6789/0 conn(0x7f5514157890 legacy=0x7f55140aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 -- 172.21.15.146:0/2474941583 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55140ad620 con 0x7f55140ace80 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f55137fe700 1 --2- 172.21.15.146:0/2474941583 >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 0x7f55140ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f550bfff700 1 --2- 172.21.15.146:0/2474941583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 0x7f5514157350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0098e0 tx=0x7f54fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2fac3c94eb01da6 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5512ffd700 1 -- 172.21.15.146:0/2474941583 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011630 con 0x7f55140ace80 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5512ffd700 1 -- 172.21.15.146:0/2474941583 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f54fc011790 con 0x7f55140ace80 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.809+0000 7f5512ffd700 1 -- 172.21.15.146:0/2474941583 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011b10 con 0x7f55140ace80 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- 172.21.15.146:0/2474941583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 msgr2=0x7f5514157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 --2- 172.21.15.146:0/2474941583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 0x7f5514157350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0098e0 tx=0x7f54fc0091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- 172.21.15.146:0/2474941583 shutdown_connections 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 --2- 172.21.15.146:0/2474941583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55140ace80 0x7f5514157350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.826 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 --2- 172.21.15.146:0/2474941583 >> v2:172.21.15.161:3300/0 conn(0x7f5514156f80 0x7f55140ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- 172.21.15.146:0/2474941583 >> 172.21.15.146:0/2474941583 conn(0x7f5514066e00 msgr2=0x7f5514067200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:27.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- 172.21.15.146:0/2474941583 shutdown_connections 2024-08-15T17:03:27.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- 172.21.15.146:0/2474941583 wait complete. 2024-08-15T17:03:27.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 Processor -- start 2024-08-15T17:03:27.827 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.813+0000 7f5519e19700 1 -- start start 2024-08-15T17:03:27.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 0x7f551416c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 0x7f55140973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551415dca0 con 0x7f5514157890 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514098820 con 0x7f55140ace80 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551415d090 con 0x7f5514156f80 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 0x7f55140973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.829 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f55137fe700 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 0x7f551416c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550bfff700 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.146:40916/0 (socket says 172.21.15.146:40916) 2024-08-15T17:03:27.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550bfff700 1 -- 172.21.15.146:0/2082596584 learned_addr learned my addr 172.21.15.146:0/2082596584 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-08-15T17:03:27.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 -- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 msgr2=0x7f5514169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 -- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 msgr2=0x7f551416c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 0x7f551416c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 -- 172.21.15.146:0/2082596584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504009420 con 0x7f5514157890 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f55137fe700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 0x7f551416c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550bfff700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5513fff700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 0x7f55140973b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5514157c80 tx=0x7f550c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00a7e0 con 0x7f5514157890 2024-08-15T17:03:27.832 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f550c00a940 con 0x7f5514157890 2024-08-15T17:03:27.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc009420 con 0x7f5514157890 2024-08-15T17:03:27.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00ace0 con 0x7f5514157890 2024-08-15T17:03:27.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.817+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551415d650 con 0x7f5514157890 2024-08-15T17:03:27.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551415de50 con 0x7f5514157890 2024-08-15T17:03:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 5) v1 ==== 80143+0+0 (secure 0 0 0) 0x7f550c007300 con 0x7f5514157890 2024-08-15T17:03:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f550b7fe700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 0x7f54f4062d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-15T17:03:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f550c07cc30 con 0x7f5514157890 2024-08-15T17:03:27.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f55137fe700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 0x7f54f4062d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-15T17:03:27.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.821+0000 7f55137fe700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 0x7f54f4062d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5504011600 tx=0x7f55040091a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-15T17:03:27.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:27.829+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+169067 (secure 0 0 0) 0x7f551415de50 con 0x7f5514157890 2024-08-15T17:03:28.041 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.029+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55140645c0 con 0x7f5514157890 2024-08-15T17:03:28.042 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.029+0000 7f550b7fe700 1 -- 172.21.15.146:0/2082596584 <== mon.0 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3540 (secure 0 0 0) 0x7f550c04d3a0 con 0x7f5514157890 2024-08-15T17:03:28.043 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-08-15T17:03:28.043 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4911,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1111982821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1111982821},{"type":"v1","addr":"172.21.15.161:6835","nonce":1111982821}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4923,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/1321381699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":1321381699},{"type":"v1","addr":"172.21.15.146:6833","nonce":1321381699}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5266,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2911946540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2911946540},{"type":"v1","addr":"172.21.15.161:6837","nonce":2911946540}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2175561360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2175561360},{"type":"v1","addr":"172.21.15.146:6835","nonce":2175561360}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[]} 2024-08-15T17:03:28.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 msgr2=0x7f54f4062d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 0x7f54f4062d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5504011600 tx=0x7f55040091a0 comp rx=0 tx=0).stop 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 msgr2=0x7f55140973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 0x7f55140973b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5514157c80 tx=0x7f550c001a20 comp rx=0 tx=0).stop 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 shutdown_connections 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:6832/24128,v1:172.21.15.161:6833/24128] conn(0x7f54f4060890 0x7f54f4062d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:28.046 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5514157890 0x7f55140973b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5514156f80 0x7f5514169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 --2- 172.21.15.146:0/2082596584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55140ace80 0x7f551416c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 >> 172.21.15.146:0/2082596584 conn(0x7f5514066e00 msgr2=0x7f55140af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 shutdown_connections 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:2024-08-15T17:03:28.033+0000 7f5519e19700 1 -- 172.21.15.146:0/2082596584 wait complete. 2024-08-15T17:03:28.047 INFO:teuthology.orchestra.run.smithi146.stderr:dumped fsmap epoch 22 2024-08-15T17:03:28.057 DEBUG:tasks.cephfs.filesystem:already dead... 2024-08-15T17:03:28.057 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-15T17:03:28.057 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-15T17:03:28.058 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.072 INFO:tasks.ceph.mds.a:Stopped 2024-08-15T17:03:28.073 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-15T17:03:28.073 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.086 INFO:tasks.ceph.mds.c:Stopped 2024-08-15T17:03:28.087 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-15T17:03:28.087 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.102 INFO:tasks.ceph.mds.b:Stopped 2024-08-15T17:03:28.102 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-15T17:03:28.102 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.118 INFO:tasks.ceph.mds.d:Stopped 2024-08-15T17:03:28.119 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-15T17:03:28.119 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-15T17:03:28.119 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.200 INFO:tasks.ceph.osd.0:Stopped 2024-08-15T17:03:28.200 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-15T17:03:28.201 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.273 INFO:tasks.ceph.osd.1:Stopped 2024-08-15T17:03:28.273 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-15T17:03:28.273 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.367 INFO:tasks.ceph.osd.2:Stopped 2024-08-15T17:03:28.367 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-15T17:03:28.368 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.440 INFO:tasks.ceph.osd.3:Stopped 2024-08-15T17:03:28.441 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-15T17:03:28.441 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.527 INFO:tasks.ceph.osd.4:Stopped 2024-08-15T17:03:28.527 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-15T17:03:28.528 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.592 INFO:tasks.ceph.osd.5:Stopped 2024-08-15T17:03:28.592 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-15T17:03:28.592 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.664 INFO:tasks.ceph.osd.6:Stopped 2024-08-15T17:03:28.664 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-15T17:03:28.664 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.736 INFO:tasks.ceph.osd.7:Stopped 2024-08-15T17:03:28.736 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-15T17:03:28.736 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-15T17:03:28.737 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.768 INFO:tasks.ceph.mgr.y:Stopped 2024-08-15T17:03:28.768 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-15T17:03:28.768 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.796 INFO:tasks.ceph.mgr.x:Stopped 2024-08-15T17:03:28.797 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-15T17:03:28.797 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-15T17:03:28.797 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.817 INFO:tasks.ceph.mon.a:Stopped 2024-08-15T17:03:28.818 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-15T17:03:28.818 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.855 INFO:tasks.ceph.mon.b:Stopped 2024-08-15T17:03:28.856 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-15T17:03:28.856 INFO:teuthology.orchestra.run:waiting for 300 2024-08-15T17:03:28.873 INFO:tasks.ceph.mon.c:Stopped 2024-08-15T17:03:28.874 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-15T17:03:28.875 DEBUG:teuthology.orchestra.run.smithi146:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-08-15T17:03:28.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:03:28.900 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-15T17:03:29.282 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:03:29.283 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-15T17:03:29.483 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:03:29.484 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-15T17:03:29.694 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:03:29.694 DEBUG:teuthology.orchestra.run.smithi146:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-15T17:03:29.891 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:03:29.892 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-15T17:03:30.289 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:03:30.289 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-15T17:03:30.504 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:03:30.505 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-15T17:03:30.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:03:30.671 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-15T17:03:30.845 INFO:tasks.ceph:Archiving mon data... 2024-08-15T17:03:30.847 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/data/mon.a.tgz 2024-08-15T17:03:30.847 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-08-15T17:03:30.854 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.pMfwcKNevZ 2024-08-15T17:03:30.855 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pMfwcKNevZ 2024-08-15T17:03:30.933 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.pMfwcKNevZ 2024-08-15T17:03:30.959 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.pMfwcKNevZ is 304KB 2024-08-15T17:03:31.018 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.pMfwcKNevZ 2024-08-15T17:03:31.024 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/data/mon.b.tgz 2024-08-15T17:03:31.025 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-08-15T17:03:31.031 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.nL5QWKVTOv 2024-08-15T17:03:31.032 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nL5QWKVTOv 2024-08-15T17:03:31.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.nL5QWKVTOv 2024-08-15T17:03:31.195 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.nL5QWKVTOv is 403KB 2024-08-15T17:03:31.260 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.nL5QWKVTOv 2024-08-15T17:03:31.267 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/data/mon.c.tgz 2024-08-15T17:03:31.267 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-08-15T17:03:31.274 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.d5nCLLNFC3 2024-08-15T17:03:31.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.d5nCLLNFC3 2024-08-15T17:03:31.373 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.d5nCLLNFC3 2024-08-15T17:03:31.442 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.d5nCLLNFC3 is 403KB 2024-08-15T17:03:31.504 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.d5nCLLNFC3 2024-08-15T17:03:31.511 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-15T17:03:31.512 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T17:03:31.514 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-15T17:03:31.529 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-15T17:03:31.529 DEBUG:teuthology.orchestra.run.smithi146:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T17:03:31.569 DEBUG:teuthology.orchestra.run.smithi161:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-15T17:03:31.610 INFO:teuthology.orchestra.run.smithi161.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T17:03:31.644 INFO:teuthology.orchestra.run.smithi146.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-15T17:03:31.651 INFO:tasks.ceph:Archiving crash dumps... 2024-08-15T17:03:31.652 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi146/crash 2024-08-15T17:03:31.653 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T17:03:31.671 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi161/crash 2024-08-15T17:03:31.672 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-15T17:03:31.687 INFO:tasks.ceph:Shutting down logrotate 2024-08-15T17:03:31.688 DEBUG:teuthology.orchestra.run.smithi146:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:31.717 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:31.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:31.775 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-15T17:03:31.813 INFO:tasks.ceph:Compressing logs... 2024-08-15T17:03:31.813 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T17:03:31.817 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-15T17:03:31.842 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33296.log 2024-08-15T17:03:31.843 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-08-15T17:03:31.843 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-08-15T17:03:31.843 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33296.log.gz 2024-08-15T17:03:31.844 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32204.log 2024-08-15T17:03:31.844 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33107.log 2024-08-15T17:03:31.845 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33794.log: /var/log/ceph/ceph-client.admin.32204.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-08-15T17:03:31.845 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32042.log 2024-08-15T17:03:31.845 INFO:teuthology.orchestra.run.smithi146.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-08-15T17:03:31.846 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32204.log.gz 2024-08-15T17:03:31.846 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29946.log 2024-08-15T17:03:31.846 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34355.log 2024-08-15T17:03:31.846 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33107.log.gz 2024-08-15T17:03:31.847 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35173.log 2024-08-15T17:03:31.847 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29946.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32042.log.gz/var/log/ceph/ceph-client.admin.34355.log: 2024-08-15T17:03:31.848 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32150.log 2024-08-15T17:03:31.848 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29946.log.gz 2024-08-15T17:03:31.848 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34355.log.gz 2024-08-15T17:03:31.848 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35173.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35173.log.gz 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31113.log 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32150.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34644.log 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30247.log 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32150.log.gz 2024-08-15T17:03:31.850 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34085.log 2024-08-15T17:03:31.851 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31113.log.gz 2024-08-15T17:03:31.854 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-15T17:03:31.854 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-15T17:03:31.854 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24240.log 2024-08-15T17:03:31.854 INFO:teuthology.orchestra.run.smithi161.stderr:gzip /var/log/ceph/ceph-mon.b.log: -5 --verbose -- /var/log/ceph/ceph-client.1.29138.log 2024-08-15T17:03:31.854 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24265.log 2024-08-15T17:03:31.855 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.29138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24371.log 2024-08-15T17:03:31.855 INFO:teuthology.orchestra.run.smithi161.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24240.log.gz 2024-08-15T17:03:31.855 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-15T17:03:31.861 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24265.log: /var/log/ceph/ceph-client.admin.24371.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.24371.log.gz 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.24265.log.gz 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.1.29138.log.gz 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24316.log 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24426.log 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24316.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24316.log.gz 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-15T17:03:31.862 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-15T17:03:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24426.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.audit.log 2024-08-15T17:03:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 88.8% 2024-08-15T17:03:31.863 INFO:teuthology.orchestra.run.smithi161.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24426.log.gz 2024-08-15T17:03:31.863 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.29427.log 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34644.log.gz 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32652.log 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34085.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30247.log.gz 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35310.log 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34085.log.gz 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35038.log 2024-08-15T17:03:31.864 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31961.log 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34768.log 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32652.log.gz 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30485.log 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35310.log.gz 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35038.log.gz 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31961.log.gz 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34002.log 2024-08-15T17:03:31.865 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34768.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34768.log.gz 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34002.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30485.log.gz 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31685.log 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34002.log.gz 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34463.log 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.31685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34490.log 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31685.log.gz 2024-08-15T17:03:31.866 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30411.log 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34463.log: 87.6%/var/log/ceph/ceph-client.admin.34490.log: -- replaced with /var/log/ceph/ceph-client.admin.34463.log.gz 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34490.log.gz 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33323.log 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32679.log 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30411.log: /var/log/ceph/ceph-client.admin.33323.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30411.log.gz 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33323.log.gz 2024-08-15T17:03:31.867 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32914.log 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32679.log.gz 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30058.log 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29703.log 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32914.log.gz 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29703.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30058.log.gz 2024-08-15T17:03:31.868 INFO:teuthology.orchestra.run.smithi146.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29703.log.gz 2024-08-15T17:03:31.872 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24399.log 2024-08-15T17:03:31.872 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.29427.log: 93.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T17:03:31.872 INFO:teuthology.orchestra.run.smithi161.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.1.29427.log.gz 2024-08-15T17:03:31.873 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.29251.log 2024-08-15T17:03:31.873 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24399.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24399.log.gz 2024-08-15T17:03:31.873 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-15T17:03:31.873 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.3.29251.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T17:03:31.873 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.b.log: 89.1% -- replaced with /var/log/ceph/ceph-client.3.29251.log.gz 2024-08-15T17:03:31.876 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-15T17:03:31.876 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.log: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T17:03:31.876 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24290.log 2024-08-15T17:03:31.883 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-15T17:03:31.883 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34382.log 2024-08-15T17:03:31.883 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-08-15T17:03:31.883 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34984.log 2024-08-15T17:03:31.883 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30031.log 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34382.log: /var/log/ceph/ceph-client.admin.34984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33161.log 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30384.log 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34382.log.gz 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34984.log.gz 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30031.log: /var/log/ceph/ceph-client.admin.33161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29850.log 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr: 89.3% 92.8% -- replaced with /var/log/ceph/ceph-client.admin.30031.log.gz -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-08-15T17:03:31.884 INFO:teuthology.orchestra.run.smithi146.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33161.log.gz 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31470.log 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35335.log 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30384.log.gz 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29850.log: /var/log/ceph/ceph-client.0.31470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35362.log 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32804.log 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35335.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29850.log.gz 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33672.log 2024-08-15T17:03:31.885 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35362.log.gz 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35335.log.gz 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29739.log 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.0.31470.log.gz 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34247.log 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32804.log.gz 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32285.log 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24247.log 2024-08-15T17:03:31.886 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33672.log.gz 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29739.log.gz 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30707.log 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32285.log: /var/log/ceph/ceph-client.admin.24247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31799.log 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34247.log.gz 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32285.log.gz 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31517.log 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30707.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.24247.log.gz 2024-08-15T17:03:31.887 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30488.log 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30707.log.gz 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31517.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31799.log.gz 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31907.log 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31517.log.gz 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33921.log 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30488.log.gz 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31853.log 2024-08-15T17:03:31.888 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31907.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30492.log 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33921.log: /var/log/ceph/ceph-client.admin.31853.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31907.log.gz 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30487.log 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33921.log.gz 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30491.log 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31853.log.gz 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30492.log: /var/log/ceph/ceph-client.admin.30487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34544.log 2024-08-15T17:03:31.889 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30487.log.gz 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31302.log 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30492.log.gz 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30491.log.gz 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33377.log 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34544.log.gz 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30708.log 2024-08-15T17:03:31.890 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31302.log: /var/log/ceph/ceph-client.admin.33377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34193.log 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33377.log.gz 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32598.log 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24130.log 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35011.log 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32598.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34193.log.gz 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.31302.log.gz 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30708.log.gz 2024-08-15T17:03:31.891 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.24130.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32598.log.gz 2024-08-15T17:03:31.892 INFO:teuthology.orchestra.run.smithi146.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.24130.log.gz 2024-08-15T17:03:31.892 INFO:teuthology.orchestra.run.smithi146.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-15T17:03:31.893 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-15T17:03:31.893 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24290.log.gz 2024-08-15T17:03:31.898 INFO:teuthology.orchestra.run.smithi161.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-15T17:03:31.899 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24343.log 2024-08-15T17:03:31.906 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.7.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-15T17:03:31.906 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.24343.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.24343.log.gz 2024-08-15T17:03:31.924 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-15T17:03:31.931 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34712.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32995.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35011.log: /var/log/ceph/ceph-client.admin.34712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32231.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35011.log.gz 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34712.log.gz 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34590.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33738.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-08-15T17:03:31.932 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32231.log: /var/log/ceph/ceph-client.admin.34590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32995.log.gz 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33711.log 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34590.log.gz 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32231.log.gz 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33738.log.gz 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32860.log 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33645.log: gzip /var/log/ceph/ceph-client.admin.33711.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35092.log 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-08-15T17:03:31.933 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32860.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33711.log.gz 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32860.log.gz 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29886.log 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33510.log 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35092.log.gz 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.29886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33867.log 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29886.log.gz 2024-08-15T17:03:31.934 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30438.log 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.0.33510.log: /var/log/ceph/ceph-client.admin.33867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32463.log 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30438.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33867.log.gz 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30139.log 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33134.log 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30438.log.gz 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32463.log: /var/log/ceph/ceph-client.admin.30139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30705.log 2024-08-15T17:03:31.935 INFO:teuthology.orchestra.run.smithi146.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32463.log.gz 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30139.log.gz 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30193.log 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30704.log 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30705.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33134.log.gz 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33765.log 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30705.log.gz 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30193.log.gz 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35146.log 2024-08-15T17:03:31.936 INFO:teuthology.orchestra.run.smithi146.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.33510.log.gz 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30301.log 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30704.log.gz 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33765.log: /var/log/ceph/ceph-client.admin.35146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31005.log 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33765.log.gz 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35146.log.gz 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35119.log 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29805.log 2024-08-15T17:03:31.937 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31005.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30301.log.gz 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30978.log 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35119.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31005.log.gz 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30357.log 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35119.log.gz 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29805.log.gz 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33242.log 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-08-15T17:03:31.938 INFO:teuthology.orchestra.run.smithi146.stderr: 88.7%/var/log/ceph/ceph-client.admin.30357.log: -- replaced with /var/log/ceph/ceph-client.admin.30978.log.gz 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33564.log 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30357.log.gz 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34957.log 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33242.log.gz 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31358.log 2024-08-15T17:03:31.939 INFO:teuthology.orchestra.run.smithi146.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33564.log.gz 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34957.log: /var/log/ceph/ceph-client.admin.30924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32490.log 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34957.log.gz 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz--verbose 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr:-- /var/log/ceph/ceph-client.admin.34301.log 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32490.log: /var/log/ceph/ceph-client.admin.33618.log 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.31358.log.gz 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.32490.log.gz 2024-08-15T17:03:31.940 INFO:teuthology.orchestra.run.smithi146.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30002.log 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34301.log.gz 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32887.log 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34328.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33618.log.gz 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35254.log 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30002.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32326.log 2024-08-15T17:03:31.941 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32887.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30002.log.gz 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35254.log: /var/log/ceph/ceph-client.admin.32326.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32887.log.gz 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30112.log 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35254.log.gz 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.1.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32326.log.gz 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32625.log 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32941.log 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30112.log: /var/log/ceph/ceph-client.admin.32625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31718.log 2024-08-15T17:03:31.942 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30112.log.gz 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32015.log 88.6% 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32625.log.gz 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32941.log: /var/log/ceph/ceph-client.admin.31718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32941.log.gz 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30486.log 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32015.log: gzip -5 --verbose 87.9%-- /var/log/ceph/ceph-client.admin.31826.log 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31718.log.gz 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30486.log: gzip 88.4%-5 -- replaced with /var/log/ceph/ceph-client.admin.32015.log.gz--verbose 2024-08-15T17:03:31.943 INFO:teuthology.orchestra.run.smithi146.stderr:-- /var/log/ceph/ceph-client.admin.34220.log 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30486.log.gz-5 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:--verbose -- /var/log/ceph/ceph-client.admin.30700.log 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31826.log: /var/log/ceph/ceph-client.admin.34220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31248.log 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31826.log.gz 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34220.log.gz 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32831.log 2024-08-15T17:03:31.944 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31248.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30700.log.gz/var/log/ceph/ceph-client.admin.35227.log: gzip 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.30274.log 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31248.log.gz 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.32831.log: -- /var/log/ceph/ceph-client.admin.34930.log 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31167.log 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32831.log.gz 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32353.log 2024-08-15T17:03:31.945 INFO:teuthology.orchestra.run.smithi146.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30274.log.gz 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34436.log 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31167.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34930.log.gz 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31086.log 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32353.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31167.log.gz 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33948.log 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32353.log.gz 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32096.log 2024-08-15T17:03:31.946 INFO:teuthology.orchestra.run.smithi146.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34436.log.gz 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32434.log 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31086.log.gz 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31059.log 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32096.log: /var/log/ceph/ceph-client.admin.32434.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33948.log.gz 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31988.log 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32096.log.gz 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32434.log.gz 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31658.log 2024-08-15T17:03:31.947 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31880.log 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31988.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31059.log.gz 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr: 88.0%/var/log/ceph/ceph-client.admin.31658.log: -- replaced with /var/log/ceph/ceph-client.admin.31988.log.gz 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31032.log 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31194.log 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31658.log.gz 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33049.log 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31032.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31880.log.gz 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32258.log 2024-08-15T17:03:31.948 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31194.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31032.log.gz 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30220.log 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33049.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31194.log.gz 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34617.log 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32258.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33049.log.gz 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34058.log 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30220.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32258.log.gz 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32712.log 2024-08-15T17:03:31.949 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34617.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30220.log.gz 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31275.log 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34617.log.gz 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34409.log 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34058.log.gz 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32712.log: /var/log/ceph/ceph-client.admin.31275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34849.log 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32712.log.gz 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32544.log 2024-08-15T17:03:31.950 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34409.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31275.log.gz 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:gzip /var/log/ceph/ceph-client.admin.34849.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34822.log 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34409.log.gz 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34849.log.gz-5 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:--verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr: 88.4%/var/log/ceph/ceph-client.admin.33455.log: -- replaced with /var/log/ceph/ceph-client.admin.32544.log.gz 2024-08-15T17:03:31.951 INFO:teuthology.orchestra.run.smithi146.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34822.log.gz 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31140.log 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33840.log 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34031.log 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31140.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34741.log 2024-08-15T17:03:31.952 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31140.log.gz 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33840.log.gz 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31221.log 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30493.log 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34741.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34031.log.gz 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33269.log 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31221.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34741.log.gz 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-08-15T17:03:31.953 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30493.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31221.log.gz 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32380.log /var/log/ceph/ceph-client.admin.33269.log: 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30493.log.gz 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33188.log 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33269.log.gz 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29913.log 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33188.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32380.log.gz 2024-08-15T17:03:31.954 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34517.log 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29913.log: gzip -5 --verbose 88.2%-- /var/log/ceph/ceph-client.admin.33350.log -- replaced with /var/log/ceph/ceph-client.admin.33188.log.gz 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33022.log 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29913.log.gz 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34517.log: /var/log/ceph/ceph-client.admin.33350.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.34517.log.gz 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:-5 --verbose -- /var/log/ceph/ceph.log 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33350.log.gz 2024-08-15T17:03:31.955 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30951.log 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33975.log 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33022.log.gz 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30166.log 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30703.log 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33975.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30951.log.gz 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33975.log.gz 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-15T17:03:31.956 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31934.log 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30166.log.gz 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.0.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.30703.log.gz 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30709.log 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32517.log 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30709.log: /var/log/ceph/ceph-client.admin.34876.log 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31934.log.gz 2024-08-15T17:03:31.957 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35281.log 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32517.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30709.log.gz 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30897.log 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32517.log.gz 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31745.log 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35281.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.34876.log.gz 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.29973.log 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30897.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35281.log.gz 2024-08-15T17:03:31.958 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31745.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.19268.log 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30897.log.gz 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.29973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33078.log 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31745.log.gz 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30328.log 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph.tmp-client.admin.19268.log: 0.0% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29973.log.gz 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.19268.log.gz 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34166.log 2024-08-15T17:03:31.959 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33894.log 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30328.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33078.log.gz 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34166.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30328.log.gz 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34671.log 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34795.log 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32571.log 2024-08-15T17:03:31.960 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33894.log.gz 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34671.log.gz 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32069.log 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32571.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34795.log.gz 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34274.log 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32571.log.gz 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 /var/log/ceph/ceph-client.admin.32069.log: --verbose -- /var/log/ceph/ceph-client.admin.30490.log 2024-08-15T17:03:31.961 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32777.log 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32069.log.gz 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30490.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34274.log.gz 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33591.log 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32777.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30490.log.gz 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35065.log 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34139.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32777.log.gz 2024-08-15T17:03:31.962 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.33591.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24389.log 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33591.log.gz 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.35065.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35065.log.gz 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32177.log 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31608.log 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.24389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32177.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24389.log.gz 2024-08-15T17:03:31.963 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.2.31608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30706.log 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32177.log.gz 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31772.log 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.34903.log: /var/log/ceph/ceph-client.admin.30706.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.2.31608.log.gz 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30706.log.gz 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32407.log 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30085.log 2024-08-15T17:03:31.964 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.31772.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-15T17:03:31.965 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.32407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31772.log.gz 2024-08-15T17:03:31.965 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-client.admin.30085.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32407.log.gz 2024-08-15T17:03:31.965 INFO:teuthology.orchestra.run.smithi146.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30085.log.gz 2024-08-15T17:03:31.965 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-15T17:03:31.968 INFO:teuthology.orchestra.run.smithi161.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-15T17:03:32.029 INFO:teuthology.orchestra.run.smithi161.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-15T17:03:32.039 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/ceph-osd.2.log: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-15T17:03:32.048 INFO:teuthology.orchestra.run.smithi161.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-15T17:03:32.064 INFO:teuthology.orchestra.run.smithi146.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-15T17:03:32.064 INFO:teuthology.orchestra.run.smithi161.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-15T17:03:32.070 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-15T17:03:32.071 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-08-15T17:03:32.071 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m0.250s 2024-08-15T17:03:32.071 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m1.080s 2024-08-15T17:03:32.072 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.069s 2024-08-15T17:03:32.092 INFO:teuthology.orchestra.run.smithi146.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-15T17:03:32.100 INFO:teuthology.orchestra.run.smithi146.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-15T17:03:32.131 INFO:teuthology.orchestra.run.smithi146.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-15T17:03:32.133 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-08-15T17:03:32.133 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m0.314s 2024-08-15T17:03:32.133 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m1.107s 2024-08-15T17:03:32.133 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.126s 2024-08-15T17:03:32.133 INFO:tasks.ceph:Archiving logs... 2024-08-15T17:03:32.134 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi146/log 2024-08-15T17:03:32.134 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T17:03:32.434 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi161/log 2024-08-15T17:03:32.434 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-15T17:03:32.583 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-15T17:03:32.606 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-15T17:03:32.606 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T17:03:32.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-15T17:03:32.681 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-15T17:03:32.691 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-15T17:03:32.691 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:03:32.694 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-15T17:03:32.717 INFO:teuthology.orchestra.run.smithi146.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:03:32.718 INFO:teuthology.orchestra.run.smithi146.stdout:============================================================================== 2024-08-15T17:03:32.718 INFO:teuthology.orchestra.run.smithi146.stdout:*hv01.front.sepi 67.205.162.81 3 u 50 64 177 0.088 0.936 0.816 2024-08-15T17:03:32.718 INFO:teuthology.orchestra.run.smithi146.stdout:+hv02.front.sepi 63.231.80.2 3 u 43 64 177 0.069 0.863 1.067 2024-08-15T17:03:32.718 INFO:teuthology.orchestra.run.smithi146.stdout:+hv03.front.sepi 74.6.168.72 3 u 40 64 177 0.090 1.370 1.376 2024-08-15T17:03:32.718 INFO:teuthology.orchestra.run.smithi146.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout:+hv01.front.sepi 67.205.162.81 3 u 43 64 177 0.168 1.115 1.739 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout:*hv02.front.sepi 63.231.80.2 3 u 49 64 177 0.099 1.159 1.775 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout:+hv03.front.sepi 74.6.168.72 3 u 41 64 177 0.119 0.451 1.284 2024-08-15T17:03:32.720 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-08-15T17:03:32.721 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-15T17:03:32.730 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-15T17:03:32.731 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-15T17:03:32.758 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-15T17:03:32.792 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-15T17:03:32.825 INFO:teuthology.task.internal:Duration was 777.987900 seconds 2024-08-15T17:03:32.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-15T17:03:32.859 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-15T17:03:32.859 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T17:03:32.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-15T17:03:32.913 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-15T17:03:32.914 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-08-15T17:03:32.914 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T17:03:32.926 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-08-15T17:03:32.926 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-15T17:03:32.938 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-15T17:03:32.938 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T17:03:32.940 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-15T17:03:32.974 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-15T17:03:32.975 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T17:03:33.017 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-15T17:03:33.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-15T17:03:33.156 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-15T17:03:33.156 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T17:03:33.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-15T17:03:33.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-15T17:03:33.200 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:03:33.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:03:33.229 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-08-15T17:03:33.231 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-08-15T17:03:33.247 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:03:33.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:03:33.254 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-15T17:03:33.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-15T17:03:33.299 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-15T17:03:33.309 INFO:teuthology.task.internal:Transferring archived files... 2024-08-15T17:03:33.310 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi146 2024-08-15T17:03:33.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T17:03:33.337 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-15_16:34:53-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7856960/remote/smithi161 2024-08-15T17:03:33.337 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-15T17:03:33.363 INFO:teuthology.task.internal:Removing archive directory... 2024-08-15T17:03:33.363 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T17:03:33.365 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-15T17:03:33.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-15T17:03:33.381 INFO:teuthology.task.internal:Not uploading archives. 2024-08-15T17:03:33.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-15T17:03:33.409 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-15T17:03:33.409 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T17:03:33.412 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-15T17:03:33.417 INFO:teuthology.orchestra.run.smithi146.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 17:03 /home/ubuntu/cephtest 2024-08-15T17:03:33.421 INFO:teuthology.orchestra.run.smithi161.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 15 17:03 /home/ubuntu/cephtest 2024-08-15T17:03:33.423 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-15T17:03:33.443 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-15T17:03:33.512 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 777.9879004955292 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f3e9e0b963554613927d2255e4156aa1 status: fail success: false 2024-08-15T17:03:33.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-15T17:03:33.588 INFO:teuthology.run:FAIL